Week #2: Time To Research

1. State Of The Art

Chat bots are known for being tools that are not usually very refined or do not offer the natural speech that would be expected, which tends to push people away from their usage.

The truth is, chat bots are difficult to implement in a convincing manner and only in recent years are chat bots starting to show some sign of intelligence and capability of adapting to the person they are talking with, mostly due to the advancements made in the field of Natural Language Processing (NLP).

As such, the team decided to spend this week searching for the state-of-the-art tools and techniques to implement a convincing and natural chat bot, that people want to use.

Nowadays, the field of NLP is mostly dominated by deep neural networks, such as RNN and LSTM, along with their variants and other more complex models.

Models such as BERT and GPT-2 are some of the most well-known and used models in the last few years in the field of NLP, having popularised the usage of Transfer Learning in this field, which has allowed chat bots to be implemented even in the absence of large amounts of data that are usually necessary to train deep neural networks.

In 2020, OpenAI released GPT-3, the successor to GPT-2, which upgraded the 1.5k million parameters found in prior version to 175k million (100x increase), being pre-trained on nearly half a trillion words and achieving SOTA performance on several NLP benchmarks without the need of fine-tuning.

GPT-3 has demonstrated that a language model trained on enough data, can solve problems that it has never encountered before, with little to no fine-tuning being necessary. In a situation, such as the one that the team is facing, where little to no data is readily avaliable to train the chat bot, the aforementioned characteristics of GPT-3 will allow the team to develop a competent and believable chat bot in a short notice.

2. Project Roadmap

As it was done in the previous challenges, the team decided to establish a set of milestones to be reached in each week of the challenge's development, although it is worth noting that the plan is prone to changes in the future:

Week #3 (14/03 - 18/03)

• Study the GPT-3 neural network and understand the best approach for the chat bot implementation;

• Gather a dataset with examples of interactions between a client and a store assistant, in order to specialise the GPT-3 neural network, through Transfer Learning;

• Gather a clothes store dataset, with a catalog of clothes and the respective attributes that can later be used in the product recommendation (e.g. color, size, category);

• Write the Introduction chapter of the scientific article.

Week #4 (21/03 - 25/03)

• Train GPT-3 neural networks to execute slot tagging and response generation based on the client's utterances;

• Train a LSTM/GRU neural network to classify the client's emotions;

• Write the chapter "State of the Art" of the scientific article.

Week #5 (28/03 - 01/04)

• Implement the search/recommendation engine in the back-end module, capable of providing the user with a set of products based on the dialogue had between him and the bot;

• Fine-tune the performance/training of the neural networks trained in the previous week;

• Write the chapter "Proposal" of the scientific article.

Week #6 (04/04 - 08/04)

• Implement a context manager in the back-end module, which manages the attributes identified by the neural network and provides recommendations formulated by the Recommendation Engine;

• Write the conclusion and abstract of the scientific article;

• Write the chapter "Development" of the scientific article, following the CRISP-DM model.

Week #7 (11/04 - 15/04)

• Implement a front-end that allows the user to interact with the chat bot via text or voice input, with the recommendations being shown as the dialogue goes on;

• Write the conclusion and abstract of the scientific article;

• Improvements on both the project and the scientific article.
  

In the following weeks' posts, the team will check on the plan and verify if everything is on track, by confronting the obtained results with the goals set in the challenge roadmap.