Course Objective:

We will discuss emerging topics in natural language processing, focusing on understanding the development and usages of large-scale language models. We will focus on four topics (learning objectives of LMs, model specialization, continual learning of LMs and human-LM interaction). This course is designed for graduate students and highly motivated undergraduates who are interested in NLP research. This course aims to teach and practice:

• Cutting-edge research in natural language processing. You will learn recent progresses and remaining challenges.

• How to formulate and evaluate NLP problems and develop solutions for them.

• How to read and criticize research papers and communicate research both orally and in writing.

Course Structure:

This is a seminar course where most class will consist of students presenting 1-2 recent research paper and leading discussions on it. We will follow role-playing paper reading seminar. Each week, you are expected to read 2-4 papers carefully.

The course is designed as an active seminar course. You will get to know your classmates, discuss papers and approaches, sometimes struggle to understand challenging papers that is hard to understand. Ideally, you should be excited about the intricacies of language understanding and willing to independently explore the literature.

Everyone are required to read the assigned paper prior to the class. A subset of the students will be assigned “presentation roles” on a rotating basis for which they will need to come prepared. The presentation roles are designed to expose students to the different aspects of being a researcher. By the end of the course, students should be comfortable reading, reviewing, implementing and extending NLP papers.

Prerequisites / Intended audiences:

This is an advanced graduate level course and assumes background in machine learning, programming and basic knowledge of natural language processing (equivalent to CS 371/CS388 course at UT). I don’t strictly enforce whether you have taken courses or not, but you should be comfortable with what’s taught in those courses. If you are looking for a lecture-based course with a structured, instructor-driven overview of NLP, this is probably not the right course for you. Consider taking CS 371 (undergraduate) or CS 388 (graduate) course. You should be comfortable digesting a research paper from ML/NLP venues.

Note: If you are unclear whether you meet these requirements, please consult the instructor in advance. Auditing is not allowed unless previously discussed with the instructor.

Logistics:

Required Materials / Equipment:

• You will need an access to laptops (to log in to zoom for online classes) and to make presentation, etc. We will not have textbook.

Grading / Workload

	% of grade	Due Dates
Class presentation	25%
Discussion Lead Sign-up		Aug 25th
Discussion Lead	15%	throughout the semester
Final Project Presentation	10%	Nov 28th, 30th
Class participation	25%
Provide feedback to classmates intermediate check-ins	7%	Nov 2th Nov 4th
Participation in-class / attendance / role-playing	18%	throughout
Mini Writing Assignment	10%	Sept 7th
Final Project	40%
One-page Check-in	5%	Sept 20thSept 28th
Intermediate Check-in (2-3 pages)	5%	Oct 31st
Final Write-up	30%	Nov 21stDec 3rd

In-class presentation (25% of grades)

Class participation (25% of grades)

Mini Writing Assignment (10% of total grade)

• This assignment must be done individually.
  • In this assignment, you will ponder about ethical/societal impact of NLP technology. Towards this end, you will do two short writing assignment:
    • Write a version of “AI/NLP Researcher’s Oath”, describing standards and principles for people developing AI/NLP technology. You can think of this something similar to Hippocratic Oath for Doctors. The format is flexible, but you should aim to cover various axis of ethical codes that’s relevant for AI researchers at this day and era. (up to 300 words).
    • You will find one article from the media discussing LLM and its impact on society and write a short commentary about it (up to 300 words). How do you think about the viewpoint presented by the journalist? Do you think they included interviews of adequate parties? What is missing in their articles?

Final Project (40% of the total grade)

Course Policy / Logistics / Communication

• All assignments are due midnight on the deadline date, to be submitted on Canvas (unless otherwise noted). You will have four slip days (each slip day is 24 hours) throughout the semester, which you can use for mini writing assignment or final project (but does not apply for course presentation or course participation (including writing assignment feedback)).

• By default, all classes will be in person and will not support online attendance. However, a few sessions will be held online (will be announced beforehand) if we have virtual guests or other special circumstances. I will not take formal attendance but attendance will count towards final class participation grade. It is okay to skip a few classes throughout the semester when you are traveling or sick (don’t show up sick), but if you did not show up for classes consistently (or do not participate in class discussions), you will not receive full credit for class participation. Also, see section on religious holy day below.

• We will use EdDiscussion for class communication (announcements, etc) and Canvas for assignment submissions.
  • Link: https://edstem.org/us/courses/42110/

Asking for help

The best way to reach the course staff is posting on EdDiscussion. If you cannot make the office hour time, I’d be happy to arrange another time if possible.

GRADE BREAKS

+/- grades will be used for the final class grade.

Topics that will be covered in this class

The paper and topic list below is tentative and will be finalized before the week of discussion. Each topic will be discussed for about 5-6 classes, and there will be 2-3 guest lectures.

Preliminary

Before starting the discussions, we will review recent papers describing architecture and training of base LMs. This will give us background to understand LLMs better for the semester.

• Llama2: Open Foundation and Fine-Tuned Chat Models (Touvron et al, META)

• Textbooks Are All You Need (Gunaseka et al, June 2023, Microsoft Research)

• Unifying Language Learning Paradigms (Tay et al 22)

• Language Models as Agent Models (Andreas et al, 2022)

• A Pretrainer’s Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity

Beyond Next Word Prediction — Learning Objective of LMs

While language models are known to be trained with ``next word prediction", most recent models are also trained with other objectives. In this part, we delve into these newer pre-training objectives such as learning from human feedback, instruction tuning, etc.

Background:

• Training language models to follow instructions with human feedback (Ouyang et al, 2022)

• Improving language models by retrieving from trillions of tokens (Borgeaud et al., 2022)

High-level Discussion:

• Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback (Casper et al)

• Yoav Goldberg's Blog Article on "Why RL?"

Data / Framework:

• LIMA: Less Is More for Alignment (Zhou et al, ArXiv 23)

• AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback

• Aligning Large Language Models through Synthetic Feedback (Kim et al ArXiv 23)

Alternative Learning Objectives (including non-RL):

• Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Rafailov et al, ArXiv 23)

Theory-of-Mind:

• Few-shot Language Coordination by Modeling Theory of Mind (Zhu et al 2023)

Specialization of LMs

The scaling law is undeniable in the recent era of language models. Can we borrow the power of large models into smaller models? We explore a few paths that have been proposed to distill bigger model into a smaller model, including symbolic distillation (where larger models are used to generate text data).

Symbolic distillation

• Symbolic Knowledge Distillation: from General Language Models to Commonsense Models (West at al, NAACL 2022)

• Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing (Jung et al, ArXiv 2023)

• Referee: Reference-Free Sentence Summarization with Sharper Controllability through Symbolic Knowledge Distillation (Sclar et al, ArXiv 2023)

• SCOTT: Self-Consistent Chain-of-Thought Distillation (Wang, ACL 23)

Specialization of LM pre-training

• Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset (Henderson et al 2022)

• Large Language Models Encode Clinical Knowledge (Singhal et al 22)

• Teaching algorithmic reasoning via In-Context Learning (Zhou et al, 22)

Adaptors / Domain specific modeling

• Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models (Li et al, ArXiv 2022)

• Towards a unified view of parameter-efficient transfer learning (He, Zhou et al, ICLR 2022)

• LoRA: Low-Rank Adaptation of Large Language Models (Hu et al, 21)

• Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference (Lei et al 23)

• Exploring the Benefits of Training Expert Language Models over Instruction Tuning (Jang et al ICML23)

High-level Discussion:

• The False Promise of Imitating Proprietary LLMs (Gudibande et al, 2023)

Adding actions to prediction space:

• Toolformer: Language Models Can Teach Themselves to Use Tools (Schick et al, ArXiv 2023)

• WebArena: A Realistic Web Environment for Building Autonomous Agents (Zhou et al, ArXiv 2023)

• WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents (Yao et al 2023)

Human - LLM interaction

In this section, we will focus on how humans will interact with emerging technology. What are emerging use cases, and how does LM shape human behaviors? We will look into both papers from HCI literature as well as from NLP/ML venues.

High level Discussion

• Evaluating Human-Language Model Interaction (Lee et al, preprint, 2023)

Information Visualization:

• Causal Perception in Question-Answering Systems (Law et al, CHI 21)

• ScatterShot: Interactive In-context Example Curation for Text Transformation (Wu et al, IUI 23)

Using LLMs

• Why Johnny Can't Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts (CHI 2023) J.D. Zamfirescu-Pereira, Richmond Wong, Bjoern Hartmann, Qian Yang

• Generative Agents: Interactive Simulacra of Human Behavior (Park et al ArXiv 2023)

• Decision-Oriented Dialogue for Human-AI Collaboration [Lin et al, ArXiv 2023]

Impact on Humans

• Co-Writing with Opinionated Language Models Affects Users’ Views (Jakesch et al, CSCW23)

• Explaining Machine Learning Models with Interactive Natural Language Conversations Using TalkToModel (Slack et al, 23)

Knowledge Augmentation / Editing

Language models function as a knowledge base, using knowledge memorized during the pretraining stage. How can we inspect the knowledge and potentially update the outdated knowledge? How can we inject new knowledge or remove outdated knowledge in LMs? This part could also touch upon causality literature (which parts of the neural network parameters contain specific knowledge?) This part will cover two paths for knowledge updating: retrieval based knowledge augmentation and parameter update methods.

Knowledge Localization

• Inspecting and Editing Knowledge Representations in Language Models (Hernandaz et al, 2023)

• Mass-Editing Memory in a Transformer (Meng et al ArXiv 2023)

• RECKONING: Reasoning through Dynamic Knowledge Encoding (Chen et al, ArXiv 23)

• Meta-Learning Online Adaptation of Language Models (Hu et al ArXiv 23)

Task-level Editing

• EDITING MODELS WITH TASK ARITHMETIC (Ilharco et al, ICLR 23)

Retrieval-based Augmentation

• Large Language Models Can Be Easily Distracted by Irrelevant Context (Shi et al, ICML 2023)

• Retrieval-Augmented Multimodal Language Modeling (Yasunaga et al., 2023)

Course Schedule

Accommodation

If you are a student with a disability, or think you may have a disability, and need accommodations please contact Disability and Access (D&A). You may refer to D&A website for contact and more information: http://diversity.utexas.edu/disability. If you are already registered with D&A, please deliver your Accommodation Letter to me as early as possible in the semester so we can discuss your approved accommodations.

The university is committed to creating an accessible and inclusive learning environment consistent with university policy and federal and state law. Please let me know if you experience any barriers to learning so I can work with you to ensure you have equal opportunity to participate fully in this course.

Honor code

The University of Texas at Austin strives to create a dynamic and engaging community of teaching and learning where students feel intellectually challenged; build knowledge and skills; and develop critical thinking, creativity, and intellectual curiosity. As a part of this community, it is important to engage in assignments, exams, and other work for your classes with openness, integrity, and a willingness to make mistakes and learn from them. The UT Austin honor code champions these principles:

I pledge, as a member of the University of Texas community, to do my work honestly, respectfully, and through the intentional pursuit of learning and scholarship.

The honor code affirmation includes three additional principles that elaborate on the core theme:

• I pledge to be honest about what I create and to acknowledge what I use that belongs to others.

• I pledge to value the process of learning in addition to the outcome, while celebrating and learning from mistakes.

• This code encompasses all of the academic and scholarly endeavors of the university community.

The honor code is more than a set of rules, it reflects the values that are foundational to your academic community. By affirming and embracing the honor code, you are both upholding the integrity of your work and contributing to a campus culture of trust and respect.

Academic Integrity

Students who violate University rules on academic misconduct are subject to the student conduct process. A student found responsible for academic misconduct may be assigned both a status sanction and a grade impact for the course. The grade impact could range from a zero on the assignment in question up to a failing grade in the course. A status sanction can range from a written warning, probation, deferred suspension and/or dismissal from the University. To learn more about academic integrity standards, tips for avoiding a potential academic misconduct violation, and the overall conduct process, please visit the Student Conduct and Academic Integrity website at: http://deanofstudents.utexas.edu/conduct.

Sharing of Course Materials is Prohibited

No materials used in this class, including, but not limited to, lecture hand-outs, videos, assessments (quizzes, exams, papers, projects, homework assignments), in-class materials, review sheets, and additional problem sets, may be shared online or with anyone outside of the class without explicit, my written permission. Unauthorized sharing of materials may facilitate cheating. The University is aware of the sites used for sharing materials, and any materials found online that are associated with you, or any suspected unauthorized sharing of materials, will be reported to Student Conduct and Academic Integrity in the Office of the Dean of Students. These reports can result in initiation of the student conduct process and include charge(s) for academic misconduct, potentially resulting in sanctions, including a grade impact.

Using Artificial Intelligence

The creation of artificial intelligence tools for widespread use is an exciting innovation. These tools have both appropriate and inappropriate uses in classwork. The use of artificial intelligence tools (such as ChatGPT) in this class:

• …is permitted for students who wish to use them, provided the content generated by AI is properly cited.

If you are considering the use of AI writing tools but are unsure if you are allowed or the extent to which they may be utilized appropriately, please ask.

For more information about AI in education, see the Center for Teaching and Learning’s “5 Things to Know about ChatGPT” webpage that includes additional suggested syllabi statements for your consideration.]

Religious Holy Days

By UT Austin policy, you must notify me of your pending absence for a religious holy day as far in advance as possible of the date of observance. If you must miss a class, an examination, a work assignment, or a project in order to observe a religious holy day, you will be given an opportunity to complete the missed work within a reasonable time after the absence.

Names and pronouns

Class rosters are provided to the instructor with the student’s legal name, unless they have added a chosen name with the registrar’s office. If you have not yet done so, I will gladly honor your request to address you with the name and pronouns that you prefer for me to use for you. It is helpful to advise me of any changes or needs regarding your name and pronouns early in the semester so that I may make appropriate updates to my records and be informed about how to support you in this class.

• For instructions on how to add your pronouns to Canvas, visit this site.

• If you would like to update your chosen name with the registrar’s office, you can do so here, and reference this guide.

• For additional guidelines prepared by the Gender and Sexuality Center for changing your name on various campus systems, see the Resources page under UT Resources here.

Important Safety Information

Students in this class should be aware of the following university policies related to Texas’ Open Carry Law:

• Students in this class who hold a license to carry are asked to review the university policy regarding campus carry.

• Individuals who hold a license to carry are eligible to carry a concealed handgun on campus, including in most outdoor areas, buildings and spaces that are accessible to the public, and in classrooms.

• It is the responsibility of concealed-carry license holders to carry their handguns on or about their person at all times while on campus. Open carry is NOT permitted, meaning that a license holder may not carry a partially or wholly visible handgun on campus premises or on any university driveway, street, sidewalk or walkway, parking lot, parking garage, or other parking area.

• Per my right, I prohibit carrying of handguns in my personal office. Note that this information will also be conveyed to all students verbally during the first week of class. This written notice is intended to reinforce the verbal notification, and is not a “legally effective” means of notification in its own right.

TITLE IX Disclosure

Beginning January 1, 2020, Texas Education Code, Section 51.252 (formerly known as Senate Bill 212) requires all employees of Texas universities, including faculty, to report to the Title IX Office any information regarding incidents of sexual harassment, sexual assault, dating violence, or stalking that is disclosed to them. Texas law requires that all employees who witness or receive information about incidents of this type (including, but not limited to, written forms, applications, one-on-one conversations, class assignments, class discussions, or third-party reports) must report it to the Title IX Coordinator. Before talking with me, or with any faculty or staff member about a Title IX-related incident, please remember that I will be required to report this information.

Although graduate teaching and research assistants are not subject to Texas Education Code, Section 51.252, they are mandatory reporters under federal Title IX regulations and are required to report a wide range of behaviors we refer to as sexual misconduct, including the types of misconduct covered under Texas Education Code, Section 51.252. Title IX of the Education Amendments of 1972 is a federal civil rights law that prohibits discrimination on the basis of sex – including pregnancy and parental status – in educational programs and activities. The Title IX Office has developed supportive ways and compiled campus resources to support all impacted by a Title IX matter.

If you would like to speak with a case manager, who can provide support, resources, or academic accommodations, in the Title IX Office, please email:  supportandresources@austin.utexas.edu. Case managers can also provide support, resources, and accommodations for pregnant, nursing, and parenting students.

For more information about reporting options and resources, please visit: https://titleix.utexas.edu, contact the Title IX Office via email at: titleix@austin.utexas.edu, or call 512-471-0419.

Campus safety

The following are recommendations regarding emergency evacuation from the Office of Emergency Management, 512-232-2114:

• Students should sign up for Campus Emergency Text Alerts at the page linked above.

• Occupants of buildings on The University of Texas at Austin campus must evacuate buildings when a fire alarm is activated. Alarm activation or announcement requires exiting and assembling outside.

• Familiarize yourself with all exit doors of each classroom and building you may occupy. Remember that the nearest exit door may not be the one you used when entering the building.

• Students requiring assistance in evacuation shall inform their instructor in writing during the first week of class.

• In the event of an evacuation, follow the instruction of faculty or class instructors. Do not re-enter a building unless given instructions by the following: Austin Fire Department, The University of Texas at Austin Police Department, or Fire Prevention Services office.

• For more information, please visit the Office of Emergency Management.

University Resources

For a list of university resources that may be helpful to you as you engage with and navigate your courses and the university, see the University Resources Students Canvas page.