Google Research tackles the most challenging problems in CS and related fields. Being bold and taking risks is essential to what we do, and research teams are embedded throughout Google, allowing our discoveries to affect billions of users each day.
The compelling benefit to researchers is that their innovations can be implemented fast and big. Googleís unique infrastructure facilitates ideasí speed to market ó allowing their ideas to be trialled by millions of users before their paper is even published.
Today weíre talking to Cathy Wong, a former Research intern. Read on!
Left: Cathy; Right: her fellow intern
Can you tell us about yourself and your masters topic?
Iím a masters student at Stanford University, where Iím a part of the Computational Vision and Geometry Lab ó I actually just joined this October, and Iím working on projects related to semantic segmentation. I also studied at Stanford as an undergrad, and previously I worked under Sebastian Thrun with Andre Esteva and Brett Kuprel on deep learning for skin cancer detection. So I work on a lot of vision projects, and Iím especially interested in projects that lie at the intersection of machine learning and healthcare. Iím also really interested in human cognition! I loved reading books by Oliver Sacks and other neuroscientists as a kid, but when I first started in computer science, I never considered that there would be much of a direct overlap where Iíd get to actually mess around in both fields. Within artificial intelligence research, though, it seems like we still have a lot to learn from actual human brains.
How did you get to work in this area?
Thereís this class at Stanford, CS231N, on deep learning for computer vision. On the very first day, I remember that the professor who co-taught the class ó Dr. Fei Fei Li ó went through this presentation, and one of the slides was about how the initial layers of convolutional neural networks learn basic edge detecting filters that actually closely parallel the basic edge detectors found in cat and human visual cortices, suggesting that there was a deeper and more fundamental connection between these two vision systems. I thought that was insane, and also insanely cool. I joined Sebastian Thrunís lab a little later, and have been working on AI research since then.
Why did you apply for an internship at Google and how supportive was your masters advisor?
Iíd heard really great things about research at Google, and even in my classes and labs, read lots of very impressive work coming from teams in Mountain View, London, and Zurich. I was hoping to get a better sense of what research looks like outside of an academic setting, and the scope of projects and expertise was a huge draw. Also, zillions of GPUs.
My masterís advisor at Stanford is Dan Jurafsky, who is the man. Heís a computer scientist and linguist, has written a book about the language of food, and is basically the best, as far as Iím concerned. He was super supportive.
What project was your internship focused on?
I worked under Andrea Gesmundo from the Applied Machine Intelligence team on Multitask Neural Model Search, a framework to automate deep learning architecture design using reinforcement learning. This work builds off of the Neural Architecture Search research done by Barrett and Quoc from the Google Brain team - that framework was one of the first to successfully apply reinforcement learning to automatically generate convolutional neural networks.
Our project focused on extending that framework so that we could automatically design architectures for multiple different tasks, simultaneously. For example, the same framework could design a model that worked well for sentiment analysis tasks, and another that worked well for language identification, at the same time.
We then showed that it was possible to transfer that framework, so that knowledge learned from designing architectures for previous tasks could be reused in totally new, unseen settings. When actual humans design machine learning models, we donít start completely from scratch every time ó we can take advantage of general intuitive design patterns weíve observed before, as well as remember what models did and didnít work on similar tasks in the past ó and this research tries to take a step closer to doing the same thing in our automated model design.
Did you publish at Google during your internship?
Yes! We submitted our work to ICML, where itís currently under review (so fingers crossed). The pre-print is also up on Arxiv.
How closely connected was the work you did during your internship to your masters topic?
Although Andrea and I discussed a bunch of project ideas in the months before the internship, this project was actually a chance to try something fairly different from my masterís research at Stanford. For me, at least, that turned out to be one of the best things about this internship ó I really loved the chance to explore a very different aspect of AI research, especially one that benefited from the guidance and computational resources available within Google, and I left with a much deeper interest in reinforcement learning that Iíve continued to explore back at Stanford.
Did you write your own code?
Heck, yeah! And then I deployed it cavalierly with enormous care across tons of GPUs. One really awesome thing about interning, though, is the chance to build off of the collaborative engineering effort of other incredibly talented engineers and researchers. I worked pretty closely with code that was being updated almost daily by researchers on the Brain team over in Mountain View, and that kind of cross-continental engineering work feels really neat.
This is your third internship at Google. What were the reasons to come back to Google Zurich?
Third timeís the charm? But actually, Iíve been lucky enough to work at a different office, on very different projects, during all three internships at Google ó after my freshman year, I worked with the Glass team in Mountain View, and later I worked in New York on Google Classroom. Each time, I left with a much deeper understanding and appreciation for that particular field, and the care and expertise each of those teams brought to those particular domains. This summer, though, I wanted to come back to work on research in particular. Both of my previous internships had been very software engineering focused, and I was excited to work on AI research that more closely parallels the work Iím excited about at Stanford.
Also, Zurich! Iíve never been to Switzerland before, and this summer, one of my fellow interns and I took a train out to hike past the Matterhorn. She wisely remembered to bring along a Toblerone bar for comparison. The real thing is much more breathtaking (but a lot less chocolatey.)
[Editorís note: the photo referenced is the photo at the beginning of this post!]
What key skills have you gained from your time at Google?
My team held a weekly reading group, where weíd gather to read and discuss cutting-edge AI papers chosen by different members of the team. This turned out to be one of the very best experiences of the internship ó it was incredibly helpful to step back and get a better sense of whatís happening within a very rapidly changing field. Listening to colleagues step through these papers helped me learn to more rigorously assess any given paper ó to ask what the experiments really mean, and how its conclusions could generalize to our own current and future projects. Those are questions that Iíve tried to ask more about any work since the summer. That commitment to keeping up with the very coolest things happening within the field also just serves to remind me, often, of what exactly I love about this work and how much there is left to tackle.
What impact has this internship experience had on your master's?
A ton. I really enjoyed diving deeply into research that was largely outside of my own masterís expertise. So much is changing within reinforcement learning right now, and Iíve definitely brought back what I learned ó and a sparked interest in related work ó to my research here.
Looking back on your experiences now: Why should a master's student apply for an internship at Google? Any advice to offer?
Thereís a kind of magical combination of people and resources that means you can work and learn so much within so short a time ó especially if you love research and havenít yet done a PhD, like myself. The internship offers that same rigor and breadth of very cool projects in a very compressed package.
When youíre here, definitely ask questions. Talk to other people about their research, because itís going to be very awesome and maybe even directly relevant. Join a reading group. Or start a reading group. And get someone to show you how to actually use the espresso machines. That milk frothy thingie? Life changing.
Florida Coastal Law's Practitioner Clinics Florida Coastal Law's Practitioner Clinic is an innovative course offering students a chance
to work closely on pro bono cases with a practicing lawyer. Students are often invited to
work on these cases in the practitioner's office off campus, and the Clinics are offered as
two credits with an evening classroom component.
Florida Coastal Law Offers LLM & Certificate in Logistics & Transportation Florida Coastal Law is the first law school in the U.S. to offer an onlineLL.M. Degree or
Certificate in Logistics & Transportation. Students in the 24-credit LL.M. or 12-credit
Certificate program gain expertise in global logistics and transportation law through
cutting-edge courses taught by attorneys and industry professionals online.