IS 698-01 / 800-01 Special Topics in Information Systems: Probabilistic Machine Learning (Fall 2021)

Information Systems Department
University of Maryland Baltimore County
Baltimore, Maryland 21250
Departmental Office: Room ITE 404, ph. 410-455-3206

Course Description

In this course, students will develop an understanding of probabilistic machine learning techniques for data analysis, and how to apply these methods in practice. Probabilistic methods, which provide a principled foundation for reasoning under uncertainty, underpin many modern machine learning methods for a broad spectrum of application domains, from text analytics, to recommender systems, to bioinformatics, and more. The course will cover modeling techniques such as directed probabilistic graphical models (a.k.a. Bayesian networks), parameter estimation methods including maximum likelihood estimation and Bayesian inference, and their application to areas such as natural language processing.

Student learning outcomes: By the end of this course, you will be able to:

Lecture time and venue: Tuesdays 4:30pm - 7:00pm online (Blackboard Collaborate)

Instructor: Dr. James Foulds
Instructor email: jfoulds [at] umbc [dot] edu. Please use Piazza for course-related questions, instead of email, so that everyone can benefit from the answers.
Instructor office hours: Tuesdays 3 - 4pm, online (Blackboard Collaborate) (other times by appointment)

Piazza: Sign up for this course at piazza.com/umbc/fall2021/is6980180001
Poll Everywhere: Vote on in-class poll questions at PollEv.com/jamesfoulds656 . Register your account for the course at PollEv.com/jamesfoulds656/register?group_key=mZQb6o2sVkLBLduWt466IPOWc , by week 2 in order to get participation credits.

Prerequisites

Required Textbook

Modeling and Reasoning with Bayesian Networks, by Adnan Darwiche (also available at the UMBC Bookstore)

We will also sometimes use Information Theory, Inference, and Learning Algorithms (Mackay, 2003), which is available as a free PDF download at the author's webpage.

Course Requirements and Grading

The project will be done in groups of 2-5 students. Project proposals are to be sent to me by email, and approved by the deadline.

In this course, participation means more than just showing up. It also refers to contributing to everyone's learning, through active engagement in peer instruction exercises, in-class discussions, and Piazza questions/answers. Participation grades will be assessed as a percentage of peer instruction questions answered (correctly or not), with a 90% response rate being sufficient for full points, and by Piazza contributions. Two or more contributions (either questions or answers) on Piazza will earn you 1% of the final grade.

With respect to final letter grades, UMBC's Catalog states that an A indicates "superior" achievement; B, "good" performance; C, "not satisfactory"; D, "unacceptable"; F, "failure." There is specifically no mention of any numerical scores associated with these letter grades. Below is how grades may be assigned based on your final points, accumulated over the semester. Grades will be assigned using a plus/minus system. It is university policy that A+, D+, and D- are not assigned. I do not grade on a curve, so that everyone in the class has the opportunity to succeed.

Final Grade Letter Grade Points when calculating GPA
91 - 100 A 4.0
89 - 90.99 A- 3.7
87 - 88.99 B+ 3.3
81 – 86.99 B 3.0
79 - 80.99 B- 2.7
77 – 78.99 C+ 2.3
71 – 76.99 C 2.0
69 - 70.99 C- 1.7
60 – 68.99 D 1.0
0 – 59.99 F 0.0

Homework and Exam Policies

Schedule

Lecture Summary Details Assessment Required reading
8/31/2021Week 1 Course overview Intro to probabilistic modeling. Motivating examples. Box's loop Bishop, C. M. (2013). Model-based machine learning. Phil. Trans. R. Soc. A, 371(1984).
9/7/2021Week 2 Probability Random variables, conditional and joint probabilities, independence, Bayes ruleHW1 out Information Theory, Inference, and Learning Algorithms (Mackay, 2003), Ch. 2
9/14/2021Week 3 Bayesian networks Bayesian networks, d-separation, Markov blankets Darwiche ch. 4
9/21/2021Week 4 Modeling with Bayesian networks MPE, MAP, modeling examples. Project brainstorming. HW1 due, HW2 out. Project groups formed by this date Darwiche ch. 5
9/28/2021Week 5 Inference in Bayesian networks Variable elimination, bucket elimination Project proposal due Darwiche ch. 6
10/5/2021Week 6 Maximum likelihood estimation Finding MLE for simple models HW2 due, HW3 out Darwiche ch. 17
10/12/2021Week 7 Bayesian inference frequentist vs Bayesian inference, MAP vs MLE, full posterior vs point estimates, posterior predictive Bishop, C. M. (2013). Model-based machine learning. Phil. Trans. R. Soc. A, 371(1984)., Ch. 3
10/19/2021Week 8 Covers up to homework 2 Midterm exam (take-home, due 11:59pm)
10/26/2021Week 9 Generative models for discrete data Conjugate priors, beta/Bernoulli, Dirichlet/multinomial, urn process interpretations, naive Bayes document model HW3 due, HW4 out Darwiche, ch. 18.4
11/2/2021Week 10 Mixture models and hidden Markov models Mixtures of Gaussians/connection to K-means, mixtures of multinomials, HMMs, Bayesian inference for HMMsProject mid-term progress report due Witten and Frank (2017), Data Mining (4th Edition), Ch 9.3 (up to Clustering with Correlated Attributes), Ch. 9.8 (up to Hidden Markov Models) (if you have it from IS733, otherwise Wikipedia articles Mixture Model, Hidden Markov Model)
11/9/2021Week 11 Topic models and mixed membership models LSA/PLSA, Genetic admixtures, LDA, collapsed Gibbs sampler HW4 due, HW5 out Blei, David M. (2012). Probabilistic topic models. Communications of the ACM, 55(4), 77-84
11/16/2021Week 12 Evaluating unsupervised models Log-likelihood on held-out data, posterior predictive checks, correlation with metadata, human evaluation Dave Blei's notes on posterior predictive checks
11/23/2021Week 13 Social network models Erdos-Renyi random graphs, stochastic blockmodels, MMSB, latent space models HW5 due Goldenberg A., Zheng, A.X., Fienberg, S.E. and Airoldi, E.M. (2010). A Survey of Statistical Network Models. Foundations and Trends® in Machine Learning: Vol. 2: No. 2 Ch. 3, can skip 3.3 and 3.7
11/30/2021Week 14 Models for computational biology Profile HMMs for protein sequence alignment. Phylogenetic models and coalescent models What is Phylogeny? article from the Tree of Life Web Project, hosted at the University of Arizona. Teh, Y. W., and D. M. Roy. (2007). Bayesian Agglomerative Clustering with Coalescents. NIPS up to Section 2.
12/7/2021Week 15Group project presentationsDigital copies of posters due. Project final report due Monday 12/13/2021 (11:59pm)
12/14/2021 Study dayNo class
12/21/2021 Exam week Take-home final exam (due 11:59pm Tues 12/21/2021)

The schedule may be subject to change. The summary and details columns are only a guideline of the content likely to be covered, and the dates on which material is covered may shift.

Online Instruction

This course will be taught online via Blackboard Collaborate, with synchronous lectures at the scheduled time. You can access lectures by navigating to the course on Blackboard, then clicking on the "Bb Collaborate" tab. Lectures will be recorded on Blackboard for later viewing, however participation during the scheduled time is expected and participation in polls during the lessons counts toward your grade.

Some of you are attending from other time zones or have other difficulties attending the class. If so, please let me know and we can discuss the possibility of alternative arrangements.

Instructional Methods

Traditional lectures will be augmented with active learning methods, primarily in the form of peer instruction exercises. Research has strongly indicated that active learning improves student outcomes in STEM fields versus traditional lecturing (Freeman et al., 2013). We will be using the Poll Everywhere service for polls and quizzes.

Pre-class reading assignments will be given for each lesson, which are very important for learning, and for making the best use of our limited time together (a partially "flipped classroom" approach). These readings are therefore required.

COVID-19 Policies

Please see this Google doc for UMBC Policies and Resources during COVID-19.

Software

This course will make extensive use of the PyMC3 python package for probabilistic machine learning.

Academic Integrity

UMBC's policies on academic integrity will be strictly enforced (see the University System of Maryland's policy document, UMBC's academic integrity overview page, the student academic conduct policy and the UMBC catalog). In particular, all of your work must be your own. Acknowledge and cite source material in your papers or assignments. While you may verbally discuss assignments with your peers, you may not copy or look at anyone else's written assignment work or code, or share your own solutions. Any exceptions will result in a zero on the assessment in question, and may lead to further disciplinary action. Some relevant excerpts from UMBC's policies, as linked to above, are:

Accessibility and Disability Accommodations, Guidance and Resources

Accommodations for students with disabilities are provided for all students with a qualified disability under the Americans with Disabilities Act (ADA & ADAAA) and Section 504 of the Rehabilitation Act who request and are eligible for accommodations. The Office of Student Disability Services (SDS) is the UMBC department designated to coordinate accommodations that would create equal access for students when barriers to participation exist in University courses, programs, or activities.

If you have a documented disability and need to request academic accommodations in your courses, please refer to the SDS website at sds.umbc.edu for registration information and office procedures.

SDS email: disAbility@umbc.edu
SDS phone: (410) 455-2459

If you will be using SDS approved accommodations in this class, please contact me (instructor) to discuss implementation of the accommodations. During remote instruction requirements due to COVID, communication and flexibility will be essential for success.

Counseling Center

Diminished mental health can interfere with optimal academic performance. The source of symptoms might be related to your course work; if so, please speak with me. However, problems with other parts of your life can also contribute to decreased academic performance. UMBC provides cost-free and confidential mental health services through the Counseling Center to help you manage personal challenges that threaten your personal or academic well-being.

Remember, getting help is a smart and courageous thing to do -- for yourself and for those who care about you. For more resources get the Just in Case mental health resources Mobile and Web App. This app can be accessed by clicking: counseling.umbc.edu/justincase.

The UMBC Counseling Center is in the Student Development & Success Center (between Chesapeake and Susquehanna Halls). Phone: 410-455-2472. Hours: Monday-Friday 8:30am-5:00pm.

Diversity Statement on Respect

Students in this class are encouraged to speak up and participate during our meetings. Because the class will represent a diversity of individual beliefs, backgrounds, and experiences, every member of this class must show respect for every other member of this class. (Statement from California State University, Chico’s Office of Diversity and Inclusion).

Family Educational Rights and Privacy Act (FERPA) Notice

Please note that as per federal law I am unable to discuss grades over email. If you wish to discuss grades, please come to my office hours.

Sexual Assault, Sexual Harassment, and Gender Based Violence and Discrimination

UMBC Policy and Federal Title IX law prohibit discrimination and harassment on the basis of sex in University programs and activities. Any student who is impacted by sexual harassment, sexual assault, domestic violence, dating violence, stalking, sexual exploitation, gender discrimination, pregnancy discrimination, gender-based harassment or retaliation should contact the University’s Title IX Coordinator to make a report and/or access support and resources:

Mikhel A. Kushner, Title IX Coordinator (she/her/hers)
410-455-1250 (direct line), kushner@umbc.edu

You can access support and resources even if you do not want to take any further action. You will not be forced to file a formal complaint or police report. Please be aware that the University may take action on its own if essential to protect the safety of the community.

If you are interested in or thinking about making a report, please see the Online Reporting Form. Please note that, while University options to respond may be limited, there is an anonymous reporting option via the online form and every effort will be made to address concerns reported anonymously.

Notice that Faculty are Responsible Employees with Mandatory Reporting Obligations:

All faculty members are considered Responsible Employees, per UMBC’s Policy on Sexual Misconduct, Sexual Harassment, and Gender Discrimination. Faculty are therefore required to report any/ all available information regarding conduct falling under the Policy and violations of the Policy to the Title IX Coordinator, even if a student discloses an experience that occurred before attending UMBC and/or an incident that only involves people not affiliated with UMBC. Reports are required regardless of the amount of detail provided and even in instances where support has already been offered or received.

While faculty members want encourage you to share information related to your life experiences through discussion and written work, students should understand that faculty are required to report past and present sexual assault, domestic and interpersonal violence, stalking, and gender discrimination that is shared with them to the Title IX Coordinator so that the University can inform students of their rights, resources and support. While you are encouraged to do so, you are not obligated to respond to outreach conducted as a result of a report to the Title IX Coordinator.

If you need to speak with someone in confidence, who does not have an obligation to report to the Title IX Coordinator, UMBC has a number of Confidential Resources available to support you:

Other Resources:

Child Abuse and Neglect:

Please note that Maryland law and UMBC policy require that I report all disclosures or suspicions of child abuse or neglect to the Department of Social Services and/or the police.

Hate, Bias, Discrimination and Harassment

UMBC values safety, cultural and ethnic diversity, social responsibility, lifelong learning, equity, and civic engagement.

Consistent with these principles, UMBC Policy prohibits discrimination and harassment in its educational programs and activities or with respect to employment terms and conditions based on race, creed, color, religion, sex, gender, pregnancy, ancestry, age, gender identity or expression, national origin, veterans status, marital status, sexual orientation, physical or mental disability, or genetic information.

Students (and faculty and staff) who experience discrimination, harassment, hate or bias or who have such matters reported to them should use the online reporting/referral form to report discrimination, hate or bias incidents. You may report incidents that happen to you anonymously. Please note that, if you report anonymously, the University’s ability to respond will be limited.


This course follows all other policy guidelines from the UMBC Office of Equity and Inclusion (OEI) listed at https://oei.umbc.edu/sample-title-ix-responsible-employee-syllabus-language/

Campus Resources