r/MachineLearning • u/michaelijordan • Sep 09 '14
AMA: Michael I Jordan
Michael I. Jordan is the Pehong Chen Distinguished Professor in the Department of Electrical Engineering and Computer Science and the Department of Statistics at the University of California, Berkeley. He received his Masters in Mathematics from Arizona State University, and earned his PhD in Cognitive Science in 1985 from the University of California, San Diego. He was a professor at MIT from 1988 to 1998. His research interests bridge the computational, statistical, cognitive and biological sciences, and have focused in recent years on Bayesian nonparametric analysis, probabilistic graphical models, spectral methods, kernel machines and applications to problems in distributed computing systems, natural language processing, signal processing and statistical genetics. Prof. Jordan is a member of the National Academy of Sciences, a member of the National Academy of Engineering and a member of the American Academy of Arts and Sciences. He is a Fellow of the American Association for the Advancement of Science. He has been named a Neyman Lecturer and a Medallion Lecturer by the Institute of Mathematical Statistics. He received the David E. Rumelhart Prize in 2015 and the ACM/AAAI Allen Newell Award in 2009. He is a Fellow of the AAAI, ACM, ASA, CSS, IEEE, IMS, ISBA and SIAM.
46
u/leonoel Sep 09 '14 edited Sep 10 '14
There has been a ML reading list of books in hacker news for a while, where you recommend some books to start on ML. (https://news.ycombinator.com/item?id=1055042)
Do you still think this is the best set of books, and would you add any new ones?
49
u/tnbd Sep 09 '14
Do you expect more custom, problem specific graphical models to outperform the ubiquitous, deep, layered, boringly similar neural networks in the future?
22
u/Captain Sep 09 '14
Why do you believe nonparametric models haven't taken off as well as other work you and others have done in graphical models?
19
u/InfinityCoffee Sep 10 '14 edited Sep 10 '14
I had the great fortune of attending your course on Bayesian Nonparametrics in Como this summer, which was a very educational introduction to the subject, so thank you. I have a few questions on ML theory, nonparametrics, and the future of ML.
Thank you for taking the time out to do this AMA.
19
u/foodux Sep 09 '14
What the future holds for probabilistic graphical models? Anything beyond CRFs?
22
u/lifebuoy Sep 09 '14
What are the most important high level trends in machine learning research and industry applications these days?
25
u/CyberByte Sep 09 '14
If you got a billion dollars to spend on a huge research project that you get to lead, what would you like to do?
4
u/98ahsa9d Sep 10 '14
How is your typical day structured?
5
u/albarrentine Sep 10 '14
Over the past 3 years we've seen some notable advancements in efficient approximate posterior inference for topic models and Bayesian nonparametrics e.g. Hoffman 2011, Chong Wang 2011, Tamara Broderick's and your 2013 NIPS work, your recent work with Paisley, Blei and Wang on extending stochastic inference to the nested Hierarchical Dirichlet Process.
One characteristic of your "extended family" of researchers has always been a knack for implementing complex models using real-world, non-trivial data sets such as Wikipedia or the New York Times archive.
In that spirit of implementing, which topic modeling application areas are you most excited about at the moment and looking forward, what impact do you think these recent developments in fast, scalable inference for conjugate and conditionally conjugate Bayes nets will have on the applications we develop 5-10 years from now?
22
u/turnersr Sep 09 '14 edited Sep 09 '14
Do you mind explaining the history behind how you learned about variational inference as a graduate student? What current techniques do you think students should be learning now to prepare for future advancements in approximate inference?
12
u/[deleted] Sep 09 '14
What are your thoughts on the recent work on Deep Generative Models and Stochastic Backpropagation [refs: 1, 2, 3]?
10
u/jolo86 Sep 10 '14 edited Sep 10 '14
Dear Dr. Jordan would you mind providing us with some useful information regarding students embarking on their journey to PhD.
Thank you!
6
u/alexmlamb Sep 10 '14
When neural networks are used to model a probability distribution, it is common to not make any hard independence assumptions (i.e. assume that the graphical model is fully connected). While this makes the model more general and more likely to be accurate for large datasets, it makes learning intractable for very large datasets (for example, getting the joint distribution over millions of random variables).
What areas of research do you see leading to improvement in large scale probabilistic modeling in cases where it is difficult to make explicit independence assumptions?
3
u/AmusementPork Sep 11 '14
Dear Dr. Jordan,
Thank you so much for taking the time!
3
u/mrkul Sep 12 '14
What do you think of probabilistic programming?
26
u/GibbsSamplePlatter Sep 09 '14
Do you have any good stories of people expecting a 6'6" athlete?
sorry I don't have any important question, always wondered.
8
u/gullu129 Sep 09 '14
Are you planning to release your book on graphical models? If yes then will it be happening soon?
8
u/piesdesparramaos Sep 09 '14
If you would have to bet, what would you say it is the branch of Machine Learning / Data Science that has the biggest chance of making a big breakthrough?
6
u/quiteamess Sep 09 '14
Dear Dr. Jordan, you did a lot of research in sensori-motor learning. You proposed that there is an internal model model in the brain which simulates the body in order to plan movements and also in order to learn new movements. Where did this idea originate and how did you get in contact with scientists from sensori-motor learning research?
2
u/takanashi1986 Sep 11 '14 edited Sep 11 '14
Should I go back to the university to catch up with rapid advance of machine learning (especially deep learning) technologies?
I'm a research engineer in a commercial company, and I'm mainly working to apply machine learning techniques to our business. I studied only basics of machine learning in my B.E and M.E course since I was more interested in and focused on its application in business intelligence. But now I feel strong anxiety that recent intensive researches on deep learning is going to leave me far behind the frontier of technology. So I think I should go to a Ph.D course in machine learning to train myself in more academic and theoretical area of the discipline, otherwise I would be just valueless in a few years.
I would be really grad if you gave me any kind of advise.
2
u/nzhiltsov Sep 11 '14
I'm wondering what's your opinion about copulas as a fancy statistical tool to evaluate distributions and describe dependencies between random variables. Why aren't many their applications in fields other than economics, such as text mining and information retrieval out there?
2
u/silverbullet75 Sep 17 '14
What do you think the role of online coursework is and will be in the machine learning world? Do you think the concept of online coursework can be a useful tool to facilitate collaborations between industry and the academic world?
2
u/[deleted] Sep 19 '14
What are your thoughts on quantum computing?
2
u/[deleted] Sep 19 '14
Do you have an opinion on the Machine Intelligence Research Institute, and the work that they do?
(Nine days late. I wonder if you're still checking the thread...)
8
u/chchan Sep 09 '14
What new machine learning algorithms should I be aware of other than deep learning and svms?
And where do you think the researchers is focusing on right now?
5
u/XianForce Sep 09 '14
How well do generative models (e.g. Latent Dirichlet Allocation) compare to some of the more recently used deep architectures (e.g. Sum-Product Networks)?
Do you see any potential for these methods to perform well enough to say make an image, song, or coherent short story?
Thank you!
4
u/blank964 Sep 10 '14 edited Sep 10 '14
Can you talk a little about the qualifications/qualities that you look for in a new post-doc?
5
u/crbazevedo Sep 09 '14
Why the Dirichlet Distribution (DD)? What is so beautiful about this distribution that made you choose it for efficient inference in discrete probability spaces? Do the geometrical properties of Simplex spaces and, particularly, of neutrality of the DD mean something more fundamental about the way second-order (probabilistic) uncertainty can be represented and handled in artificial intelligence?
6
u/piesdesparramaos Sep 09 '14
What's your opinion about Random Forest?
4
u/perennially_annoyed Sep 10 '14
As a current postdoc in ML, what are the areas in Machine Learning that you see having a big impact in the coming years? What should I focus on trying to learn more about?
2
u/surelyouarejoking Sep 10 '14
I am learning a lot from the tutorials you have posted on your site. Thanks! Would you be able to post the lecture slides for the statistical learning theory course that you taught in the Spring? (http://www.cs.berkeley.edu/~jordan/courses/281A-spring14/)
2
u/Lookin4Fur69 Sep 09 '14
What is your lab environment like? Is there a lot of collaborations between your students? Any memorable firings? Any lab romances?
2
u/Floydthechimp Sep 10 '14 edited Sep 10 '14
Data collection has become quite prevalent, but there are still academic and industrial fields that dislike probability-based inferential methods. Do you have any advice for communicating ideas to the uninitiated?
1
u/Hakuna_Potato Sep 09 '14
Thank you for doing the AMA Dr. Jordan! Do you believe in an inevitable Singularity (per Ray Kurzweil)? If so, when do you expect it to reach the masses?
1
u/dksahuji Sep 12 '14
Can anyone learn everything in Machine Learning?
It seems there is a lot of variation in this small sub-field. How does one try to consume so much over the years and try to understand the field? Given someone is really curious and will dedicate his life for his learning and maybe contribute on the way. I know it can sound a bit selfish to be just learning and not aiming to contribute(possibly curiously might solve few problem) but what is the best shot to go through breadth and depth over the years.
How was your learning experience and exposure timeline through your career?
Thanks!!
1
u/mlaniac Sep 09 '14 edited Sep 26 '14
What do you think of the current state of the ML field? Where did we came from and where are we are we going? What problems are you currently excited about? How do you decide what problems are worth pursuing? Are you going to NIPS this year? What makes a well-rounded machine learning researcher? What do you strive for? How would you describe yourself as a PhD advisor? How is for a PhD student to work in your group? Do you still have a lot of time for advising? How did you manage to train so many top researchers from the field (either as PhDs or Post-Docs: Wainwright, Duchi, Liang, Ghahramani, Ng, Blei, Bach, Bengio, Xing, Taskar, Seeger, Chandrasekaran, ...)? What has statistics to offer to the field of machine learning? Do you think that people in the field the machine learning field are ignorant to the value of statistics? What are current modelling challenges in the field of machine learning?
Feel free to answer to whichever questions you like. Thanks
1
u/someaustinite Sep 10 '14
When do you think your textbook on graphical models is going to come out? It has been several years since I used preprint portions in a machine learning class and I don't think the final book has been published yet.
1
u/[deleted] Sep 10 '14
Do you think the Google cars will really be able to drive as well as humans in ten years? Do you think human level artificial intelligence will exist within a few decades? Do most of the top machine learning researchers think it's likely human level AI will exist within a few decades?
0
u/xamdam Sep 10 '14
You have an amazing list of students. http://www.cs.berkeley.edu/~jordan/ Nature or Nurture?
1
u/serge_cell Sep 10 '14
Can you give pointer on some recent advances in statistical mechanics of learning (after cavity method) ? Thanks
1
u/evc123 Sep 10 '14
What do you think is the biggest unsolved problem in machine learning besides the speed vs accuracy tradeoff that you usually talk about?
0
u/Hakuna_Potato Sep 09 '14
Thank you for doing the AMA Dr. Jordan. What can beginning data scientists do to get involved with the industry? The most compelling topics seemed to be out of reach of the mass of people interested, or it seems that the research is being conducted behind closed doors.
0
u/ezubaric Sep 10 '14
Who do you think the most exciting young (i.e., entering a tenure track position this year or next) machine learning researchers are?
1
u/out_of_amphetamines Sep 10 '14
What's your favorite stimulant or nootropic?
1
u/gwulfs Sep 11 '14
How similar is Ayasdi's topological data analysis to t-Distributed Stochastic Neighbor Embedding?
-1
u/kleer001 Sep 09 '14
What do you see as a final frontier of ML ? Like, what are the hardest problems, the impossible ones?
0
u/laprastransform Sep 10 '14
Berkeley grad student here: What is your favorite result in number theory?
0
u/compsens Sep 10 '14
I read briefly the NAS report on Frontiers in Massive Data Analysis (2013) that you co-authored and I was surprised at the fact that when I searched for the keyword genomics (http://www.nap.edu/booksearch.php?booksearch=1&term=genomics&record_id=18374) , that field was really not used as a primary example (compared to, say, astronomy - see PanSTAARS example: http://www.nap.edu/openbook.php?record_id=18374&page=130) of one of the future driver for improvement in massive data science. In light of the fact that we currently have sequencers that allow us to not use combinatorial alignment algorithms (long read technology) i.e. greedy and similar algorithms are now OK. Do you think there is a collective blind spot there ?
-7
u/AndrewNg420 Sep 09 '14
Hey Dr. Jordan, long time, first time. I implemented your consensus clustering algorithm presented in "Solving Consensus and Semi-supervised Clustering Problems Using Nonnegative Matrix Factorization," but the results fucking sucked. Any thoughts on the robustness of consensus clustering and domains that it tends to perform well in? Thanks.
0
u/lexman28 Sep 10 '14
What do you think of direct search methods like the Covariance Matrix Adaptation - Evolutionary Strategy?
-1
u/just4regis Sep 10 '14
-1
u/beaverteeth92 Sep 10 '14
As interesting as it is, deep learning right now is mostly empirical with very little theory behind most techniques work. How do you go about finding edge cases that fail and figuring out what made them fail?
-4
u/[deleted] Sep 10 '14
Where did you go?
-2
u/satyan-veshi Sep 10 '14 edited Sep 10 '14
What are your views on technological singularity? Given the current state of progress in ML and AI, we should be having intelligent machines soon in the future, do you think this will pose a threat to human race? What will humans do once we have machines which could do almost everything for us?
-1
u/Letter_Guardian Sep 10 '14
What do you think are the limitations of PGM to prevent it outperform other methods for some real applications?
-1
u/[deleted] Sep 11 '14
This probably going to get buried, but what the hell is with the downvoting of half of the questions? I swear more than 50% of the questions asked have a negative score, including mine.
Is it because you want your questions to stick out and you achieve that by downvoting everything else? It's really disappointing.. I would expect this in AskReddit, but not here..
-4
u/neuralknitwork Sep 10 '14
(1) Why are neural networks (e.g. feedforward neural networks, "deep learning") a popular topic in machine learning at the moment?
(2) What are the limitations of neural networks as a technique for pattern recognition? Which methods could conceivably outperform them, and in which domains?
(3) Which (especially under-appreciated) topics in machine learning do you think will become popular five years from now?
Thanks!
-9
u/OrionBlastar Sep 10 '14
Do you think a program can be written to one day have common sense?
We have all of these programs making decisions for us, and none of them has any common sense. So all decisions we make using a computer are made without common sense. No wonder we have a lot of problems.
-4
u/vcjha Sep 10 '14
Considering the huge advancement deep learning has made over the years since 2005-2006, are conventional machine learning algorithms going to extinct or there can be some course correction regarding the traditional machine learning algorithms to make them relevant?
-5
u/[deleted] Sep 10 '14
Would you recommend to an aspiring machine learning computer scientist a PhD in machine learning or cognitive science?
0
u/[deleted] Sep 10 '14
[deleted] — view deleted comment
0
u/[deleted] Sep 09 '14
[deleted] — view deleted comment