Schedule

(đź“Ť CSB 480, Mudd Building) 10/11/2024, 1:00 PM - 3:00 PM
Ziyang Li (UPenn)
Relational Programming with Foundation Models
Abstract Neurosymbolic programming combines the otherwise complementary worlds of deep learning and symbolic reasoning. It thereby enables more accurate, interpretable, and domain-aware solutions to AI tasks. Scallop is a general-purpose language and compiler toolchain for developing neurosymbolic applications. A Scallop program specifies a suitable decomposition of an AI task’s computation into separate learning and reasoning modules. Learning modules are built using existing machine learning frameworks and range from custom neural models to foundation models for language, vision, and multi-modal data. Reasoning modules are specified in a declarative logic programming language based on Datalog which supports expressive reasoning patterns and probabilistic programming over structured relations. In this talk, we demonstrate relational programming in Scallop with foundation models for applications that span the domains of vision, natural language, and program analysis.

Bio: Ziyang Li is a final year PhD student from the University of Pennsylvania, advised by Prof. Mayur Naik. His primary research fields are programming languages and machine learning. He specifically focuses on Neuro-Symbolic methods, which systematically bridges state-of-the-art machine learning systems and traditional symbolic reasoning programs. He develops Scallop, a language for neuro-symbolic programming. Since its creation, the language has been applied to fields of computer vision (CV), natural language processing (NLP), security and program analysis, planning, medicine, and bio-informatics. His works have been recognized in top conferences such as PLDI, S&P, USENIX Security, NeurIPS, ICLR, and ICML.

Previous Talks

4/11/2024, 3:00 PM - 4:00 PM
Changshu Liu (UIUC)
Can Large Language Models Reason About Code?
Abstract Large Language Models (LLMs) have been widely used to automate programming tasks. Their capabilities have been evaluated by assessing code quality through test execution. However, as we will show, success in code synthesis does not imply code reasoning, which is essential to trust LLMs with tasks that involve program analysis, e.g., test generation and debugging. Therefore, we proposed a framework designed to gauge the code reasoning abilities of LLMs through several inductive reasoning tasks. CodeMind currently supports three tasks: Independent Execution Reasoning (IER), Dependent Execution Reasoning (DER), and Specification Reasoning (SR). Our extensive evaluation of ten LLMs across five benchmarks in two different programming languages for two code generation tasks (code synthesis and translation) shows that LLMs, to some degree, can explain the program execution flow, specifically for simple programs and the ones they can correctly generate. However, their performance drops for code with higher complexity, non-trivial logical and arithmetic operators, non-primitive types, and API calls. We observe that, while correlated, code generation abilities do not imply code reasoning: ranking LLMs based on test passing can be very different compared to code reasoning.

Bio: Changshu Liu is a first-year Ph.D student advised by Prof. Reyhaneh Jabbarvand. His is working on the intersection between software engineering and artificial intelligence.
3/28/2024, 3:00 PM - 4:00 PM
Alex Gu (MIT)
AI4Code, Code Modeling
Abstract

Bio: Alex is a Ph.D. student advised by Prof. Armando Solar-Lezama, and he has been working on several great projects in the area of AI4Code. You might have heard a few of his papers, such as CRUXEval, StarCoder, StarCoder2, LeanDojo, etc.
2/8/2024, 3:00 PM - 4:00 PM
Toufique Ahmed (UC Davis)
Evolution and Prospects of Large Language Models in Software Engineering
Abstract Large Language Models (LLMs) have revolutionized the field of Natural Language Processing (NLP) in recent years. These models are increasingly prominent in software development tasks, including code completion, program repair, code summarization, and incident management. We have witnessed significant changes and the introduction of innovative concepts such as few-shot learning, chain-of-thought, automatic prompt augmentation (ASAP), and multilingual training. In this presentation, I will provide a brief overview of these concepts and how LLMs have evolved in the context of comment generation tasks. I will also explore future possibilities in this area. Additionally, I will delve into the practical applications of LLMs, particularly in critical scenarios like incident management, and gain insights from incident owners regarding their perspectives on this technology.

Bio: Dr. Toufique Ahmed is currently a post doc in the University of California, Davis. His current research endeavors are centered around advancing Large Language Models (LLMs) to achieve superior performance in Software Engineering. This pursuit involves a dual focus on enhancing both the performance and trustworthiness of LLMs. Positioned at the nexus of machine learning and software engineering, his work aims to utilize the potential of machine learning to reduce the challenges faced by software developers, thereby boosting their productivity. Under the guidance of Prof. Prem Devanbu, his contributions have been recognized and featured in leading Software Engineering venues, such as ICSE, ESEC/FSE, ASE, TSE, and EMSE. His research is dedicated to forging new pathways that combine the strengths of machine learning and software engineering to facilitate improvements in software development practices. More about Dr. Ahmed can be found at: https://toufiqueparag.github.io/toufique.github.io
1/25/2024
Jacob Laurel (UIUC)
Automatic Differentiation
Abstract Jacob Laurel's research on Automatic Differentiation

Bio: Jacob Laurel is a final year PhD student in the Department of Computer Science at the University of Illinois at Urbana-Champaign. Jacob's current research focuses on static analysis of programming languages that expose continuous computations, which includes both probabilistic and differentiable programming languages. You can find more information about Jacob at: https://jsl1994.github.io

Seminar Host: Columbia ARiSE Lab led by Professor Baishakhi Ray.

Welcome to contact yrbding [at] cs.columbia.edu .

Website template from the Stanford MLSys Seminar Series.