Some of the questions we will be thinking about at #SciFM24. Feel free to add yours in the comments section.
1. What are some characteristics of problems that make them suitable to be described by foundation models?
2. How do we conceive of and construct foundation models for problems in which no clear “building blocks” exist (in contrast to areas such as materials science)?
3. Is it possible for foundation models to produce Nobel prize-type discovery of new phenomena as this usually goes against conventional wisdom? Would it be able to produce a hypothesis contrary to most data that it is likely to see?
4. What [really] is emergence? How do we better better understand the factors that determine emergent properties in large models?
5. What are the scaling laws to describe the dependency on factors such as training data size, feature dimension, computational resources, and structure and size of model?
6. Methods for training models when the inference objectives are abstract, ill-specified, or misaligned
7. Methods to develop trust in Foundation Models
8. Mathematical challenges and solution strategies for training large models given limited energy, computation, and memory
9. Mathematical characterization of the effect of data imbalance on accuracy and generalizability, especially with regard to exceedingly rare events and other phenomena in the tails of the training distribution.
10. What are the new mathematical and statistical frameworks that embody human-like properties of reasoning, insight, creativity, and critical judgment that could enable models to postulate well beyond the data used to train them?
The SciFM24 Conference is here! Join us on April 2 & 3, 2024, from 8:30 am-5:30 pm to learn about scientific foundation models and their applications from scientists and industry experts. For more information, please visit micde.umich.edu/SciFM24.
Professor at UC Irvine
6moI will definitely organize one Jamesina. Thanks a lot for all you are doing for the NRSM and URSI.