2nd workshop of AARMS CRG on Mathematical Foundations of Scientific Machine Learning

---

Welcome to the workshop

This is the 2nd workshop of the AARMS Collaborative Research Group on Mathematical Foundations in Scientific Machine Learning, which will be held on the Fredericton Campus of the University of New Brunswick on Jul 31 - Aug 01, 2023.

---

Theme

The theme of the workshop is as broad as the scientific research programs of the partners in the AARMS CRG group, including Machine Learning approaches in Mathematics, Geophysics, Chemistry, Statistics, etc.

---

Location

The workshop will take place on the Fredericton Campus of the University of New Brunswick on the unsurrendered and unceded traditional Wolastoqey land. UNB campus is a 30mins walk from the downtown area. All sessions will take place in Room 215 of the Kinesiology building, 90 Mackay Drive in Fredericton, NB. A map of the Campus can be found here.

---

Travel

Fredericton has no train station, so your best options to get to Fredericton are

• Driving
• Flying into YFC via YUL, YYZ, YTZ or YOW. There are taxis at the airport that can bring you to campus/downwtown.

Participants are requested to book accommodations themselves. Good options can be found

• Stay@UNB [campus residence]
• Carriage House Inn [B&B, 15min walk]
• Crowne Plaza - Lord Beaverbrook hotel[Downtown, 25min walk]
• Fredericton Inn [Commercial district, 40min walk]
• Hilton[Downtown hotel, 25min walk]

---

Wifi

There is eduroam on campus, and guest wifi access can be obtained via Public@UNB

---

Local organizers

• Stijn De Baerdemacker [unb,chem/math&stats]
• Nicholas Touikan [unb,math&stats],
• Viqar Husain [unb,math&stats].

---

Registration

Participation is free, but registration is required here. Registration is now closed

---

Participants

in person

• Moulay Akhloufi [U de Moncton]
• Claire Anderson [UNB]
• Amir Ayati [UNB]
• Matt Brannock [UNB]
• Dave Burns [UNB]
• Benjamin Cook [UNB]
• Paul Cook [UNB]
• Vivek Das [UNB]
• Stijn De Baerdemacker [UNB]
• Norman Deschamps [VeroSource Solutions]
• Amer El Samman [UNB]
• Syed Eqbal Alam [UNB]
• Augusto Gerolin [uOttawa]
• Elham Ghamari [UNB]
• Ehsan Ghasempouri [UNB]
• Max Hennick [TrojAI/UNB]
• Lam Ho [Dal]
• Incé Husain [UNB]
• Viqar Husain [UNB]
• Mostafa Javaheri [UNB]
• Lucas Jensen [VeroSource Solutions]
• Shermin Khosravi [UNB]
• Peter Lelievre [MtA]
• Mohammad Mehabadi Mohammadi [UNB]
• Dmytro Popovych [MUN/imath]
• Sanjeev Seahra [UNB]
• Arya Shahrostambeik [UNB]
• Dhirendra Shukla [UNB]
• Jon Sensinger [UNB]
• Nicholas Touikan [UNB]
• Rodrigo Vargas-Hernandez [McMaster]
• Saeed Vatankhah [MtA]
• Colin West [BlueSkyData]

remotely

• Mohammad Amirian [Dal]
• Ayda Azad Khorasani [MUN]
• Alex Bihlo [MUN]
• Ibrahima Dione [U de Moncton]
• Zahra Hossein Esfahani [MUN]
• Gerry Harris-Pink [MUN]
• Ronald Haynes [MUN]
• Anna Ignaszak [UNB]
• Mahsan Irajifar [Ferdowsj U]
• Chun Keat Khor [UNB]
• Scott MacLachlan [MUN]
• Rebecca Mulder [UNB]
• Robert Santacruz Zaragoza [UNB]
• Terrence Tricco [MUN]
• Hamid Usefi [MUN]
• Andy Wan [UNBC]

---

Remote Participation

Remote participants can tune in via the provided MSTeams links:

• Mon Jul 31 link
• Tue Aug 01 link

---

Program

Last update: Jul 30, 2023

time	Mon Jul 31	Tue Aug 01
08h45	Welcome
09h00	Paul Cook| 9h45 Welcome	Augusto Gerolin
10h00	coffee	coffee
10h30	Jon Sensinger	Rodrigo Vargas-Hernandez
11h30	Peter Lelievre	Amer El Samman
12h00	Dmytro Popovych	Incé Husain
12h30	lunch	lunch
14h00	Colin West	Moulay Akhloufi
14h30	Max Hennick
15h00	coffee	coffee
15h30	Panel Discussion	Lam Ho
16h00		Claire Anderson
16h30		Syed Eqbal Alam
17h00	closing	closing

---

Paul Cook [cancelled]

• affiliation: UNB
• time: Mon, 09h00-10h00
• title: Cross-lingual models in natural language processing with examples from multiword expressions and phishing detection.
• abstract: Natural language processing (NLP) seeks to build computational models of human language, primarily focused on text, for applications such as automatic translation and question answering. Cross-lingual models in NLP can leverage knowledge from one language to make predictions in another. This can enable training supervised methods for tasks in languages for which training data is unavailable. Similarly, when language-specific training data is available, incorporating additional training data from other languages can give improvements. This talk considers cross-lingual models in two areas: identifying, and predicting the compositionality of, multiword expressions (e.g., idioms such as "hit the road" and noun compounds such as "night owl") and phishing detection.

---

Jon Sensinger

• affiliation: UNB
• time: Mon, 10h30-11h30
• title: Applications of control-theoretic and computational approaches to rehabilitation engineering
• abstract: This presentation will give an overview of some of the research happening at the Institute of Biomedical Engineering within the domain of rehabilitation engineering that benefits from control-theoretic and computational approaches. It will briefly survey mechatronic prosthesis design, talk about the application of phase variables in control of powered prostheses and exoskeletons; briefly talk about machine learning applied to small-data data-sets to control upper limb prostheses, and then focus on a computational approach to understanding human-machine interactions. This latter part of the talk will explore value functions, optimal feedback control, and curiosity-driven control approaches.

---

Peter Lelievre

• affiliation: Mount Allison
• time: Tue, 11h30-12h00
• title: An investigation of regularization and constraints in surface-geometry inversion
• abstract: Typical minimum-structure mesh-based inversion methods tend to recover smoothed physical property distributions. This conflicts with geologists' interpretations about the Earth, which typically involve distinct rock units with contacts between them. While focussing and other regularization methods can help, working on a mesh is always inconsistent with the way geologists think of the Earth. Hence, we have been developing a fundamentally different inversion approach that works directly with surface-based representations of the Earth volume of interest, where surfaces of tessellated triangles represent the interfaces between rock units. We call this approach surface-geometry inversion (SGI). SGI effectively takes some initial surface-based model and alters the position of the surfaces to improve the fit to the geophysical data.
  It is important to know whether the solutions obtained from SGI are unique and stable and, if they are not, how to add regularization or constraints to make them so. Without a well-posed problem, any interpretations of the subsurface based on those solutions, and any exploration decisions based on those interpretations, are unreliable. Assessing the numerical characteristics of SGI problems is challenging because we use global heuristic optimization methods and stochastic sampling in their solution, they are severely nonlinear, and they lack explicit matrix operators and derivatives. A critical aspect is understanding when regularization/stabilization should be incorporated into the SGI optimization problem to create a well-posed problem. In this work, we make headway towards a better understanding of these issues.
  The extent to which our SGI problems are well-posed likely depends on the number of data, the resolving capability of the data type and survey geometry, the signal-to-noise ratio, the tessellation of the surface-based models, the subdivision strategy, and the size and shape of the anomalous targets. These are difficult questions to address: general rules are difficult to develop with rigorous mathematics given the nature of our SGI problems and solution methods. Hence, in this preliminary stage we develop an empirical methodology to assess the numerical characteristics of some specific, common exploration scenarios.

---

Dmytro Popovych

• affiliation: MUN/Institute of Mathematics of NAS of Ukraine
• time: Mon, 12h00-12h30
• title: Physics-informed neural networks with hard constraints
• abstract: Physics-informed neural networks (PINNs) have become an intensively researched method for solving initial boundary value problems for differential equations. This method relies on the function approximation capabilities of feedforward neural networks and measures the success of an approximation by evaluating its residual in a number of points. In most recent papers, initial and boundary constraints are represented as additional components of composite loss function to be minimized by the training process. This creates a multi-task learning problem posing significant challenges for optimization. There are earlier papers however using the approach within which the neural network is trained to satisfy the differential equation but initial value and boundary conditions are set by a fixed, non-trainable term. We call this hard constraints approach. We experiment with it using popular PDE test cases such as Allen-Cahn and KdV equations.
  We also examine approaches suitable for training accurate PINN approximations. In this context, we discuss the problem of accurate benchmarks for PINN models.
  The results were obtained in collaboration with Alex Bihlo and Roman Popovych

---

Colin West

• affiliation: BlueSky
• time: Mon, 14h00-14h30
• title: Moving Targets!
• abstract: Practice vs. Theory in data science; the absence of laurels to rest on and the dynamic requirements of the career. (Trying to) practically keep up with modern methods during exponential improvement.

---

Max Hennick

• affiliation: TrojAI/UNB
• time: Mon, 14h30-15h00
• title: A World of Words: A Brief Exploration of the Behaviour of Large Language Models
• abstract: Large language models have recently been catapulted into the mainstream with the release of ChatGPT and later GPT4. These tools are massively transformative, but much of the information surrounding them in popular culture tends to be misleading and overly speculative. In this talk we will go over what these things really are, how they suddenly became so widespread, and what we really know about their internal mechanisms.

---

Panel Discussion

• time: Mon, 15h30-17h30
• moderator: Nicholas Touikan
• participants:
  • Paul Cook [Academia]
  • Colin West [Industry]
  • Max Hennick [Academia/Industry]
  • Dhirendra Shukla [Academia/Industry]
• title: "SMARTECH: Unveiling the path to AI mastery - Skills for a thriving career in Industry."
• abstract: Panel discussion on the skills that graduates in machine learning and artificial intelligence need to have successful careers in industry. ... and what does SMARTECH stand for?

---

Augusto Gerolin

• affiliation: uOttawa
• time: Tue, 9h00-10h00
• title: Optimal Transport in Data Sciences: theory and algorithms
• abstract: In the last 30 years, the theory of optimal transportation has emerged as a fertile field of inquiry, and a diverse tool for exploring applications within and beyond mathematics, in such diverse fields as economics, statistics, biology, chemistry, quantum physics and machine learning. Optimal Transport is a natural tool to compare in a geometrically faithful way probability distributions, and very robust in performing supervised learning and unsupervised learning tasks.
  This talk will present an overview of recent advances in optimal transport from both theory and algorithm standpoint. Particular focus will be given on the entropy penalized optimal transport, in order to find a different approach to the (Kantorovich) duality. In particular, this duality approach provides an alternative proof of the convergence of the Sinkhorn algorithm with two marginals optimal transport and show convergence of the Sinkhorn algorithm in the multi-marginal case.

---

Rodrigo Vargas-Hernandez

• affiliation: McMaster
• time: Tue, 10h30-11h30
• title: Differentiable Physics: Harnessing the Power of Automatic Differentiation for physical simulations
• abstract: Contemporary statistical methods in chemistry have embraced the power of graph convolutional neural networks (GCNNs) to learn complex chemical models. However, the interpretability of these models has become a challenge, often dubbed the “black box problem.” In our work, we address this issue, focusing on SchNet neural net, by showing the feasibility of extracting from the network a compact explainable chemical model that presents some important features of chemistry. These features include chemical moiety modelling and organizing molecules in terms of their similarity using simple Euclidean distance. We do this by probing the neural activations of the graph’s nodes (representing atoms in SchNet) with different inputs. Remarkably, we find nodal responses tend to fall into categories representing the various chemical subspecies that exist (chemical moieties). The species are also organized in a manner that is chemically logical, similar structures next to each other. We also find highly resolute boundaries between the chemical species found and evidence of fractal structures in the model, that similar reaction pathways are modelled in the same direction, using the same constant vector. Lastly, we present the capability of using this internal model to transfer learn to other chemical properties such as pKa and NMR. By harnessing the power of GCNNs, we pave the way for interpretable and efficient chemical modeling, fostering a deeper understanding of machine learning models and complex chemical systems.

---

Amer El Samman

• affiliation: UNB
• time: Tue, 11h30-12h00
• title: Explainable Neural Network Chemistry
• abstract: Contemporary statistical methods in chemistry have embraced the power of graph convolutional neural networks (GCNNs) to learn complex chemical models. However, the interpretability of these models has become a challenge, often dubbed the “black box problem.” In our work, we address this issue, focusing on SchNet neural net, a graph-convolutional model of fully connected nodes, each node representing an atom-in-a-molecule with a vector of neural activations, called an embedding vector. We probe this model with molecular inputs and analyze the nodal responses, i.e. the embedding vectors. From this, we show the feasibility of extracting from the nodal GCNN some important features of chemistry. Remarkably, we find that the embedding vectors (nodal responses) tend to fall into categories representing the various chemical subspecies that exist (chemical moieties). The responses are also organized in a manner that is geometrically well-defined, similar structures giving similar embedding vectors. We also find highly resolute boundaries between the embedding vectors, by using Linear Discriminant Analysis, we can categorize all embeddings with functional groups to an error of 1e-5. There is evidence of fractal structure in the model, that is, similar reaction pathways are modelled using the same changes in the nodal activations/embeddings. Lastly, we present the capability of using this internal model to transfer learn to other chemical properties such as pKa and NMR. By harnessing the power of GCNNs, we pave the way for interpretable and efficient chemical modeling, fostering a deeper understanding of machine learning models and complex chemical systems.

---

Incé Husain

• affiliation: UNB
• time: Tue, 12h00-12h30
• title: Identifying chemical information in neural network “SchNet" 's representation of molecules
• abstract: The neural network “SchNet” has captivated computational chemists due to its highly accurate predictions about molecules. The representations of molecules that SchNet uses to compute these predictions, encoded in its “embedding vectors”, have been shown to align with traditional chemical understandings of "functional groups" - a set of atoms that describe a molecule's behaviour. However, it is unknown whether the information in the embedding vectors that underlies this is related to traditional chemical quantities. In this study, we are investigating the following question: Do the contents of SchNet’s embedding vectors relate to traditional chemical quantities? To address this, the information in embedding vectors was compared to the chemical quantity of electron densities. These representations were visualized and statistically compared. Results show that embedding vectors contain information about electron densities, but that they are not fully described by them. Ideas for further study to interpret machine learning representations of molecules will be discussed.

---

Moulay Akhloufi

• affiliation: Université de Moncton
• time: Tue, 14h00-15h00
• title: AI-Driven Solutions for Environmental Applications and Healthcare
• abstract: This presentation delves into the innovative use of AI, deep learning, and transformers in environmental applications and healthcare. The focus is on wildfire management, bioacoustics and biovision for environmental monitoring. The session showcases how AI-driven models enable early detection and real-time monitoring of wildfires, enhancing decision-making and mitigation strategies. In bioacoustics and biovision, AI facilitates accurate analysis of species diversity and ecosystem dynamics, aiding conservation efforts. The presentation also explores AI's integration into healthcare, demonstrating its potential in diagnostics, and personalized treatment, leading to improved patient outcomes and cost reduction. The presentation highlights the work conducted at the Perception, Robotics, and Intelligent Machines Lab (Université de Moncton) and shows the potential of AI to create a sustainable future for the environment and to enhance human health.

---

Lam Ho

• affiliation: Dalhousie
• time: Tue, 15h30-16h00
• title: Epidemic Forecasting using Delayed Time Embedding
• abstract: Forecasting the future trajectory of an outbreak plays a crucial role in the mission of managing emerging infectious disease epidemics. Compartmental models, such as the Susceptible-Exposed-Infectious-Recovered (SEIR), are the most popular tools for this task. They have been used extensively to combat many infectious disease outbreaks including the current COVID-19 pandemic. One downside of these models is that they assume that the dynamics of an epidemic follow a pre-defined dynamical system which may not capture the true trajectories of an outbreak. Consequently, the users need to make several modifications throughout an epidemic to ensure their models fit well with the data. However, there is no guarantee that these modifications can also help increase the precision of forecasting.
  In this talk, I will introduce a new method for predicting epidemics that does not make any assumption on the underlying dynamical system. Our method combines sparse random feature expansion and delay embedding to learn the trajectory of an epidemic.

---

Claire Anderson

• affiliation: UNB
• time: Tue, 16h00-16h30
• title: Machine Learning Towards Automated Discovery of Organic Molecules for use in Non-aqueous Redox Flow Batteries
• abstract: With the increasing demand for energy and the resources needed to provide this energy, redox flow batteries have shown potential as highly flexible large-scale energy storage systems. These electrochemical energy storage systems make use of redox processes for the conversion of electrochemical energy and are highly appealing due to the ease of their scalability to fit the size necessary for different energy storage applications. For this project, interest lies in the discovery of organic species as redox active materials for non-aqueous redox flow batteries. Machine learning has been applied to automate the process of generating different organic molecules from given parameters followed by attempts to improve the dataset through the application of a genetic algorithm (GA). The initial set of molecules was generated based on an unsubstituted bipyridine structure through a series of choices made regarding the number of nitrogen atoms, double bonds, substituents, and their positions throughout each ring as well as the linkers that connect the two ring structures. These molecules were then studied computationally to determine their reduction and cell potential, solvation free energy, and stability in order to rank these molecules to select a population to be run through the GA. Selected results from the automated generation of the initial set of molecules as well as those generated through the GA will be presented.

---

Syed Eqbal Alam

• affiliation: UNB
• time: Tue, 16h30-17h00
• title: A Communication-Efficient Local Differentially Private Algorithm in Federated Optimization
• abstract: Federated optimization, wherein several agents in a network collaborate with a central server to achieve optimal social cost over the network with no requirement for exchanging information among agents, has attracted significant interest from the research community. In this context, agents demand resources based on their local computation. Due to the exchange of optimization parameters such as states, constraints, or objective functions with a central server, an adversary may infer sensitive information of agents. We develop a differentially-private additive-increase and multiplicative-decrease algorithm to allocate multiple divisible shared heterogeneous resources to agents in a network. The developed algorithm provides a differential privacy guarantee to each agent in the network. The algorithm does not require inter-agent communication, and the agents do not need to share their cost function or their derivatives with other agents or a central server; however, they share their allocation states with a central server that keeps track of the aggregate consumption of resources. The algorithm incurs very little communication overhead; for m heterogeneous resources in the system, the asymptotic upper bound on the communication
  complexity is O(m) bits at a time step. Furthermore, if the algorithm converges in K time steps, then the upper bound communication complexity will be O(mK) bits. The algorithm can find applications in several areas, including smart cities, smart energy systems, resource management in the sixth generation (6G) wireless networks with privacy guarantees, etc. We present experimental results to check the efficacy of the algorithm. Furthermore, we present empirical analyses for the trade-off between privacy and algorithm efficiency.

---