What is the exploration vs exploitation trade off in reinforcement learning?

A key challenge that arises in reinforcement learning (RL) is the trade-off between exploration and exploitation. This challenge is unique to RL and doesn’t arise in supervised or unsupervised learning.

Exploration is any action that lets the agent discover new features about the environment, while exploitation is capitalizing on knowledge already gained. If the agent continues to exploit only past experiences, it is likely to get stuck in a suboptimal policy. On the other hand, if it continues to explore without exploiting, it might never find a good policy.

An agent must find the right balance between the two so that it can discover the optimal policy that yields the maximum rewards.

Frequently asked questions: AI tools

How does generative AI work?: Generative AI technology typically uses large language models (LLMs), which are powered by neural networks—computer systems designed to mimic the structures of brains. These LLMs are trained on a huge quantity of data (e.g., text, images) to recognize patterns that they then follow in the content they produce.

For example, a chatbot like ChatGPT generally has a good idea of what word should come next in a sentence because it has been trained on billions of sentences and “learned” what words are likely to appear, in what order, in each context.

This makes generative AI applications vulnerable to the problem of hallucination—errors in their outputs such as unjustified factual claims or visual bugs in generated images. These tools essentially “guess” what a good response to the prompt would be, and they have a pretty good success rate because of the large amount of training data they have to draw on, but they can and do go wrong.
Who owns the copyright of ChatGPT outputs?: According to OpenAI’s terms of use, users have the right to use outputs from their own ChatGPT conversations for any purpose (including commercial publication).

However, users should be aware of the potential legal implications of publishing ChatGPT outputs. ChatGPT responses are not always unique: different users may receive the same response.

Furthermore, ChatGPT outputs may contain copyrighted material. Users may be liable if they reproduce such material.
Is ChatGPT biased?: ChatGPT can sometimes reproduce biases from its training data, since it draws on the text it has “seen” to create plausible responses to your prompts.

For example, users have shown that it sometimes makes sexist assumptions such as that a doctor mentioned in a prompt must be a man rather than a woman. Some have also pointed out political bias in terms of which political figures the tool is willing to write positively or negatively about and which requests it refuses.

The tool is unlikely to be consistently biased toward a particular perspective or against a particular group. Rather, its responses are based on its training data and on the way you phrase your ChatGPT prompts. It’s sensitive to phrasing, so asking it the same question in different ways will result in quite different answers.
What is information extraction?: Information extraction refers to the process of starting from unstructured sources (e.g., text documents written in ordinary English) and automatically extracting structured information (i.e., data in a clearly defined format that’s easily understood by computers). It’s an important concept in natural language processing (NLP).

For example, you might think of using news articles full of celebrity gossip to automatically create a database of the relationships between the celebrities mentioned (e.g., married, dating, divorced, feuding). You would end up with data in a structured format, something like MarriageBetween(celebrity₁,celebrity₂,date).

The challenge involves developing systems that can “understand” the text well enough to extract this kind of data from it.
What is knowledge representation and reasoning?: Knowledge representation and reasoning (KRR) is the study of how to represent information about the world in a form that can be used by a computer system to solve and reason about complex problems. It is an important field of artificial intelligence (AI) research.

An example of a KRR application is a semantic network, a way of grouping words or concepts by how closely related they are and formally defining the relationships between them so that a machine can “understand” language in something like the way people do.

A related concept is information extraction, concerned with how to get structured information from unstructured sources.
Can ChatGPT summarize text?: Yes, you can use ChatGPT to summarize text. This can help you understand complex information more easily, summarize the central argument of your own paper, or clarify your research question.

You can also use Scribbr’s free text summarizer, which is designed specifically for this purpose.
Can ChatGPT paraphrase text?: Yes, you can use ChatGPT to paraphrase text to help you express your ideas more clearly, explore different ways of phrasing your arguments, and avoid repetition.

However, it’s not specifically designed for this purpose. We recommend using a specialized tool like Scribbr’s free paraphrasing tool, which will provide a smoother user experience.
Can ChatGPT feedback my college essay?: Yes, you use ChatGPT to help write your college essay by having it generate feedback on certain aspects of your work (consistency of tone, clarity of structure, etc.).

However, ChatGPT is not able to adequately judge qualities like vulnerability and authenticity. For this reason, it’s important to also ask for feedback from people who have experience with college essays and who know you well.

Alternatively, you can get advice using Scribbr’s essay editing service.
Can ChatGPT write my college essay?: No, having ChatGPT write your college essay can negatively impact your application in numerous ways. ChatGPT outputs are unoriginal and lack personal insight.

Furthermore, Passing off AI-generated text as your own work is considered academically dishonest. AI detectors may be used to detect this offense, and it’s highly unlikely that any university will accept you if you are caught submitting an AI-generated admission essay.

However, you can use ChatGPT to help write your college essay during the preparation and revision stages (e.g., for brainstorming ideas and generating feedback).
Is artificial intelligence (AI) the same as machine learning (ML)?: Although the terms artificial intelligence and machine learning are often used interchangeably, they are distinct (but related) concepts:

Artificial intelligence is a broad term that encompasses any process or technology aiming to build machines and computers that can perform complex tasks typically associated with human intelligence, like decision-making or translating.

Machine learning is a subfield of artificial intelligence that uses data and algorithms to teach computers how to learn and perform specific tasks without human interference.

In other words, machine learning is a specific approach or technique used to achieve the overarching goal of AI to build intelligent systems.
What is the difference between machine learning and traditional programming?: Traditional programming and machine learning are essentially different approaches to problem-solving.

In traditional programming, a programmer manually provides specific instructions to the computer based on their understanding and analysis of the problem. If the data or the problem changes, the programmer needs to manually update the code.

In contrast, in machine learning the process is automated: we feed data to a computer and it comes up with a solution (i.e. a model) without being explicitly instructed on how to do this. Because the ML model learns by itself, it can handle new data or new scenarios.

Overall, traditional programming is a more fixed approach where the programmer designs the solution explicitly, while ML is a more flexible and adaptive approach where the ML model learns from data to generate a solution.
What is an example of a machine learning application in real life?: A real-life application of machine learning is an email spam filter. To create such a filter, we would collect data consisting of various email messages and features (subject line, sender information, etc.) which we would label as spam or not spam. We would then train the model to recognize which features are associated with spam emails. In this way, the ML model would be able to classify any incoming emails as either unwanted or legitimate.
Is using ChatGPT ethical?: ChatGPT and other AI writing tools can have unethical uses. These include:

Reproducing biases and false information

Using ChatGPT to cheat in academic contexts

Violating the privacy of others by inputting personal information

However, when used correctly, AI writing tools can be helpful resources for improving your academic writing and research skills. Some ways to use ChatGPT ethically include:

Following your institution’s guidelines

Critically evaluating outputs

Being transparent about how you used the tool
Can I publish text written by ChatGPT?: According to OpenAI’s terms of use, users have the right to reproduce text generated by ChatGPT during conversations.

However, publishing ChatGPT outputs may have legal implications, such as copyright infringement.

Users should be aware of such issues and use ChatGPT outputs as a source of inspiration instead.
When should I use supervised learning?: Supervised learning should be used when your dataset consists of labeled data and your goal is to predict or classify new, unseen data based on the patterns learned from the labeled examples.

Tasks like image classification, sentiment analysis, and predictive modeling are common in supervised learning.
When should I use unsupervised learning?: Unsupervised learning should be used when your data is unlabeled and your goal is to discover the inherent structure or pattern in the data.

This approach is helpful for tasks like clustering, association, and dimensionality reduction.
What is the difference between classification and regression in supervised machine learning?: In classification, the goal is to assign input data to specific, predefined categories. The output in classification is typically a label or a class from a set of predefined options.

In regression, the goal is to establish a relationship between input variables and the output. The output in regression is a real-valued number that can vary within a range.

In both supervised learning approaches the goal is to find patterns or relationships in the input data so we can accurately predict the desired outcomes. The difference is that classification predicts categorical classes (like spam), while regression predicts continuous numerical values (like age, income, or temperature).
What is generative art?: Generative art is art that has been created (generated) by some sort of autonomous system rather than directly by a human artist. Nowadays, the term is commonly used to refer to images created by generative AI tools like Midjourney and DALL-E. These tools use neural networks to create art automatically based on a prompt from the user (e.g., “an elephant painted in the style of Goya”).

However, the term has been in use since before this technology existed, and it can also refer to any technique use by an artist (or writer, musician, etc.) to create art according to a process that proceeds autonomously—i.e., outside of the artist’s direct control. Examples of generative art that does not involve AI include serialism in music and the cut-up technique in literature.
What are some real-life applications of reinforcement learning?: Some real-life applications of reinforcement learning include:

Healthcare. Reinforcement learning can be used to create personalized treatment strategies, known as dynamic treatment regimes (DTRs), for patients with long-term illnesses. The input is a set of clinical observations and assessments of a patient. The outputs are the treatment options or drug dosages for every stage of the patient’s journey.

Education. Reinforcement learning can be used to create personalized learning experiences for students. This includes tutoring systems that adapt to student needs, identify knowledge gaps, and suggest customized learning trajectories to enhance educational outcomes.

Natural language processing (NLP). Text summarization, question answering, machine translation, and predictive text are all NLP applications using reinforcement learning.

Robotics. Deep learning and reinforcement learning can be used to train robots that have the ability to grasp various objects , even objects they have never encountered before. This can, for example, be used in the context of an assembly line.
What is deep reinforcement learning?: Deep reinforcement learning is the combination of deep learning and reinforcement learning.

Deep learning is a collection of techniques using artificial neural networks that mimic the structure of the human brain. With deep learning, computers can recognize complex patterns in large amounts of data, extract insights, or make predictions, without being explicitly programmed to do so. The training can consist of supervised learning, unsupervised learning, or reinforcement learning.

Reinforcement learning (RL) is a learning mode in which a computer interacts with an environment, receives feedback and, based on that, adjusts its decision-making strategy.

Deep reinforcement learning is a specialized form of RL that utilizes deep neural networks to solve more complex problems. In deep reinforcement learning, we combine the pattern recognition strengths of deep learning and neural networks with the feedback-based learning of RL.
Are algorithms the same as computer programs?: Algorithms and computer programs are sometimes used interchangeably, but they refer to two distinct but interrelated concepts.

An algorithm is a step-by-step instruction for solving a problem that is precise yet general.

Computer programs are specific implementations of an algorithm in a specific programming language. In other words, the algorithm is the high-level description of an idea, while the program is the actual implementation of that idea.
Are algorithms the same as artificial intelligence (AI)?: Algorithms and artificial intelligence (AI) are not the same, however they are closely related.

Artificial intelligence is a broad term describing computer systems performing tasks usually associated with human intelligence like decision-making, pattern recognition, or learning from experience.

Algorithms are the instructions that AI uses to carry out these tasks, therefore we could say that algorithms are the building blocks of AI—even though AI involves more advanced capabilities beyond just following instructions.
What is an algorithm in computer science?: In computer science, an algorithm is a list of unambiguous instructions that specify successive steps to solve a problem or perform a task. Algorithms help computers execute tasks like playing games or sorting a list of numbers. In other words, computers use algorithms to understand what to do and give you the result you need.
Why do we need algorithms?: Algorithms are valuable to us because they:

Form the basis of much of the technology we use in our daily lives, from mobile apps to search engines.

Power innovations in various industries that augment our abilities (e.g., AI assistants or medical diagnosis).

Help analyze large volumes of data, discover patterns and make informed decisions in a fast and efficient way, at a scale humans are simply not able to do.

Automate processes. By streamlining tasks, algorithms increase efficiency, reduce errors, and save valuable time.
How accurate are AI detectors?: AI detectors aim to identify the presence of AI-generated text (e.g., from ChatGPT) in a piece of writing, but they can’t do so with complete accuracy. In our comparison of the best AI detectors, we found that the 10 tools we tested had an average accuracy of 60%. The best free tool had 68% accuracy, the best premium tool 84%.

Because of how AI detectors work, they can never guarantee 100% accuracy, and there is always at least a small risk of false positives (human text being marked as AI-generated). Therefore, these tools should not be relied upon to provide absolute proof that a text is or isn’t AI-generated. Rather, they can provide a good indication in combination with other evidence.
Can I have ChatGPT write my paper?: No, it’s not a good idea to do so in general—first, because it’s normally considered plagiarism or academic dishonesty to represent someone else’s work as your own (even if that “someone” is an AI language model). Even if you cite ChatGPT, you’ll still be penalized unless this is specifically allowed by your university. Institutions may use AI detectors to enforce these rules.

Second, ChatGPT can recombine existing texts, but it cannot really generate new knowledge. And it lacks specialist knowledge of academic topics. Therefore, it is not possible to obtain original research results, and the text produced may contain factual errors.

However, you can usually still use ChatGPT for assignments in other ways, as a source of inspiration and feedback.
Can I create citations using ChatGPT?: No, it is not possible to cite your sources with ChatGPT. You can ask it to create citations, but it isn’t designed for this task and tends to make up sources that don’t exist or present information in the wrong format. ChatGPT also cannot add citations to direct quotes in your text.

Instead, use a tool designed for this purpose, like the Scribbr Citation Generator.

But you can use ChatGPT for assignments in other ways, to provide inspiration, feedback, and general writing advice.
How does ChatGPT work?: ChatGPT is a chatbot based on a large language model (LLM). These models are trained on huge datasets consisting of hundreds of billions of words of text, based on which the model learns to effectively predict natural responses to the prompts you enter.

ChatGPT was also refined through a process called reinforcement learning from human feedback (RLHF), which involves “rewarding” the model for providing useful answers and discouraging inappropriate answers—encouraging it to make fewer mistakes.

Essentially, ChatGPT’s answers are based on predicting the most likely responses to your inputs based on its training data, with a reward system on top of this to incentivize it to give you the most helpful answers possible. It’s a bit like an incredibly advanced version of predictive text. This is also one of ChatGPT’s limitations: because its answers are based on probabilities, they’re not always trustworthy.
Who owns ChatGPT?: ChatGPT is owned by OpenAI, the company that developed and released it. OpenAI is a company dedicated to AI research. It started as a nonprofit company in 2015 but transitioned to for-profit in 2019. Its current CEO is Sam Altman, who also co-founded the company.

In terms of who owns the content generated by ChatGPT, OpenAI states that it will not claim copyright on this content, and the terms of use state that “you can use Content for any purpose, including commercial purposes such as sale or publication.” This means that you effectively own any content you generate with ChatGPT and can use it for your own purposes.

Be cautious about how you use ChatGPT content in an academic context. University policies on AI writing are still developing, so even if you “own” the content, you’re often not allowed to submit it as your own work according to your university or to publish it in a journal. AI detectors may be used to detect ChatGPT content.
Who created ChatGPT?: ChatGPT was created by OpenAI, an AI research company. It started as a nonprofit company in 2015 but became for-profit in 2019. Its CEO is Sam Altman, who also co-founded the company. OpenAI released ChatGPT as a free “research preview” in November 2022. Currently, it’s still available for free, although a more advanced premium version is available if you pay for it.

OpenAI is also known for developing DALL-E, an AI image generator that runs on similar technology to ChatGPT.
What does “ChatGPT” stand for?: GPT stands for “generative pre-trained transformer,” which is a type of large language model: a neural network trained on a very large amount of text to produce convincing, human-like language outputs. The Chat part of the name just means “chat”: ChatGPT is a chatbot that you interact with by typing in text.

The technology behind ChatGPT is GPT-3.5 (in the free version) or GPT-4 (in the premium version). These are the names for the specific versions of the GPT model. GPT-4 is currently the most advanced model that OpenAI has created. It’s also the model used in Bing’s chatbot feature.
How can I use AI writing tools?: AI writing tools can be used to perform a variety of tasks.

Generative AI writing tools (like ChatGPT) generate text based on human inputs and can be used for interactive learning, to provide feedback, or to generate research questions or outlines.

These tools can also be used to paraphrase or summarize text or to identify grammar and punctuation mistakes. You can also use Scribbr’s free paraphrasing tool, summarizing tool, and grammar checker, which are designed specifically for these purposes.
Can I use AI tools to write my essay?: Using AI writing tools (like ChatGPT) to write your essay is usually considered plagiarism and may result in penalization, unless it is allowed by your university. Text generated by AI tools is based on existing texts and therefore cannot provide unique insights. Furthermore, these outputs sometimes contain factual inaccuracies or grammar mistakes.

However, AI writing tools can be used effectively as a source of feedback and inspiration for your writing (e.g., to generate research questions). Other AI tools, like grammar checkers, can help identify and eliminate grammar and punctuation mistakes to enhance your writing.
Are ChatGPT conversations private?: ChatGPT conversations are generally used to train future models and to resolve issues/bugs. These chats may be monitored by human AI trainers.

However, users can opt out of having their conversations used for training. In these instances, chats are monitored only for potential abuse.
Does ChatGPT store conversations?: OpenAI may store ChatGPT conversations for the purposes of future training. Additionally, these conversations may be monitored by human AI trainers.

Users can choose not to have their chat history saved. Unsaved chats are not used to train future models and are permanently deleted from ChatGPT’s system after 30 days.
Is there a ChatGPT app?: The official ChatGPT app is currently only available on iOS devices. If you don’t have an iOS device, only use the official OpenAI website to access the tool. This helps to eliminate the potential risk of downloading fraudulent or malicious software.
Can I use ChatGPT as a conversation partner to learn a language?: Yes, using ChatGPT as a conversation partner is a great way to practice a language in an interactive way.

Try using a prompt like this one:

“Please be my Spanish conversation partner. Only speak to me in Spanish. Keep your answers short (maximum 50 words). Ask me questions. Let’s start the conversation with the following topic: [conversation topic].”
How can I use ChatGPT for my term paper or bachelor thesis?: You can use ChatGPT to assist in the writing process for your research paper, thesis, or dissertation in the following ways:

Developing a research question

Creating an outline

Generating literature ideas

Paraphrasing text

Getting feedback
How can I detect AI writing?: Tools called AI detectors are designed to label text as AI-generated or human. AI detectors work by looking for specific characteristics in the text, such as a low level of randomness in word choice and sentence length. These characteristics are typical of AI writing, allowing the detector to make a good guess at when text is AI-generated.

But these tools can’t guarantee 100% accuracy. Check out our comparison of the best AI detectors to learn more.

You can also manually watch for clues that a text is AI-generated—for example, a very different style from the writer’s usual voice or a generic, overly polite tone.
What is the best summarizer tool?: Our research into the best summary generators (aka summarizers or summarizing tools) found that the best summarizer available in 2023 is the one offered by QuillBot.

While many summarizers just pick out some sentences from the text, QuillBot generates original summaries that are creative, clear, accurate, and concise. It can summarize texts of up to 1,200 words for free, or up to 6,000 with a premium subscription.

Try the QuillBot summarizer for free
Can deep learning models be biased in their predictions?: Deep learning models can be biased in their predictions if the training data consist of biased information. For example, if a deep learning model used for screening job applicants has been trained with a dataset consisting primarily of white male applicants, it will consistently favor this specific population over others.
What kind of data are needed for deep learning?: Deep learning requires a large dataset (e.g., images or text) to learn from. The more diverse and representative the data, the better the model will learn to recognize objects or make predictions. Only when the training data is sufficiently varied can the model make accurate predictions or recognize objects from new data.
What are ChatGPT prompts?: ChatGPT prompts are the textual inputs (e.g., questions, instructions) that you enter into ChatGPT to get responses.

ChatGPT predicts an appropriate response to the prompt you entered. In general, a more specific and carefully worded prompt will get you better responses.
What makes a good ChatGPT prompt?: A good ChatGPT prompt (i.e., one that will get you the kinds of responses you want):

Gives the tool a role to explain what type of answer you expect from it

Is precisely formulated and gives enough context

Is free from bias

Has been tested and improved by experimenting with the tool
Is ChatGPT free?: Yes, ChatGPT is currently available for free. You have to sign up for a free account to use the tool, and you should be aware that your data may be collected to train future versions of the model.

To sign up and use the tool for free, go to this page and click “Sign up.” You can do so with your email or with a Google account.

A premium version of the tool called ChatGPT Plus is available as a monthly subscription. It currently costs $20 and gets you access to features like GPT-4 (a more advanced version of the language model). But it’s optional: you can use the tool completely free if you’re not interested in the extra features.
How long will ChatGPT be free?: It’s not clear whether ChatGPT will stop being available for free in the future—and if so, when. The tool was originally released in November 2022 as a “research preview.” It was released for free so that the model could be tested on a very large user base.

The framing of the tool as a “preview” suggests that it may not be available for free in the long run, but so far, no plans have been announced to end free access to the tool.

A premium version, ChatGPT Plus, is available for $20 a month and provides access to features like GPT-4, a more advanced version of the model. It may be that this is the only way OpenAI (the publisher of ChatGPT) plans to monetize it and that the basic version will remain free. Or it may be that the high costs of running the tool’s servers lead them to end the free version in the future. We don’t know yet.
How much does ChatGPT cost?: ChatGPT is currently free to use. You just have to sign up for a free account (using your email address or your Google account), and you can start using the tool immediately. It’s possible that the tool will require a subscription to use in the future, but no plans for this have been announced so far.

A premium subscription for the tool is available, however. It’s called ChatGPT Plus and costs $20 a month. It gets you access to features like GPT-4 (a more advanced version of the model) and faster responses. But it’s entirely optional: you only need to subscribe if you want these advanced features.
When was ChatGPT released?: ChatGPT was publicly released on November 30, 2022. At the time of its release, it was described as a “research preview,” but it is still available now, and no plans have been announced so far to take it offline or charge for access.

ChatGPT continues to receive updates adding more features and fixing bugs. The most recent update at the time of writing was on May 24, 2023.
How do I access ChatGPT?: You can access ChatGPT by signing up for a free account:

Follow this link to the ChatGPT website.

Click on “Sign up” and fill in the necessary details (or use your Google account). It’s free to sign up and use the tool.

Type a prompt into the chat box to get started!

A ChatGPT app is also available for iOS, and an Android app is planned for the future. The app works similarly to the website, and you log in with the same account for both.

Questions?

Ask our team

Want to contact us directly? No problem. We are always here for you.

Frequently asked questions

See all

How does Scribbr help students graduate?: Our team helps students graduate by offering:

A world-class citation generator

Plagiarism Checker software powered by Turnitin

Innovative Citation Checker software

Professional proofreading services

Over 300 helpful articles about academic writing, citing sources, plagiarism, and more
What type of documents does Scribbr proofread?: Scribbr specializes in editing study-related documents. We proofread:

Essays

Papers

Theses

PhD dissertations

Research proposals

Personal statements

Admission essays

Motivation letters

Reports

Reflection papers

Journal articles

Capstone projects
What plagiarism and AI detection software does Scribbr use?: Scribbr’s Plagiarism Checker is powered by elements of Turnitin’s Similarity Checker, namely the plagiarism detection software and the Internet Archive and Premium Scholarly Publications content databases.

The add-on AI detector is powered by Scribbr’s proprietary software.
Which citation software does Scribbr use?: The Scribbr Citation Generator is developed using the open-source Citation Style Language (CSL) project and Frank Bennett’s citeproc-js. It’s the same technology used by dozens of other popular citation tools, including Mendeley and Zotero.

You can find all the citation styles and locales used in the Scribbr Citation Generator in our publicly accessible repository on Github.