I recall seeing posts from Deep Dream researchers' (Leon A. Gatys, Alexander S. Ecker and Matthias Bethge) a few months ago and thinking someone should turn this into an API or web application to enable people to combine their own photographs with styles of various artists. So when I saw deepart.io launched by Łukasz Kidzinski and Michał Warchoł I knew I had to try out the service and connect with the founders. Below they have graciously answered my questions about deepart.io and their own professional journey.
Mike(M): What work is deepart.io derived from?
deepart.io is based on deep learning - multi-layer neural networks which are supposed to mimic human brain. The project uses a pre-trained neural network for detecting geometric shapes in the image. The algorithm extracts the geometric shapes from the images and applies texture from the 'style' image, typically a painting by an artist, to the content image provided by the user.
M: What steps were necessary to convert these algorithms from an academic work into a service?After the publication of the original Gatys, Ecker and Berthge paper many implementations appeared on-line. As it was associated with Deep Dream project at Google, deep learning enthusiasts tried to write the code and run it on their computers. The problem was that the technology, while having the potential to attract ordinary users, was used only in the tech community. We thought of bringing it to the public by using cloud computing.
The main obstacle is that processing of a picture takes a lot of GPU time, which leads to high cost of servers. But it turned out that some users are willing to cover that cost and we managed to build sustainable, partly free web application.
M: I love that you are combining math and art. I could imagine that some people find it strange. What are some of the reactions you are getting from people?
Most of the users are amazed with the results. Unlike the Deep Dream project, in deepart we produce something which they are already familiar with - everybody knows paintings of Van Gogh.
M: What do you see in the future for deepart.io?
There are dozens of directions in which we can go - for the moment our approach is to react to users needs. In the short run, mobile applications and hi-resolutions prints of the deeparts are our main priority. Later, we would like to put energy in further developments of the algorithm, speed it up and maybe join styles of the artists.
M: As I understand it this is a side-project for both of you. Tell us about your main line of work? What do you do day-to-day?
We both work as researchers at universities [Łukasz is at École polytechnique fédérale de Lausanne and Michał is at Université catholique de Louvain] so deepart is not totally detached. As we experiment with technology, it doesn't take us that much time. As soon as we see more commercial applications, we may consider devoting more time to the project.
M: Please share some milestones in your academic and business career.
Łukasz: I recently got a grant (~$350,000 USD) for a learning platform in which we are trying to suggest best activities for users based on their psychological features. We are comparing the choice of activities based on educational research with choice suggested by machine learning methods. Business-wise, I was the founder of a platform for Polish high-school students, where they could sell their materials to other students. Over the course of 4 years the website made a net profit of around $100,000 USD.
Michał: I received a grant to conduct research in statistics of extreme events. I focus on developing statistical methodology for modeling dependence of rare events. Another milestone was my recent research stay at the Department of Statistics at Columbia University for which I received a personal travel grant. During the visit I was able to make further advancements and broaden my scientific network and collaboration.
M: Where do you each see your own academic research impacting the world?
Łukasz: I believe in the Agile methodology in research. Fast prototyping and fast iterations, borrowed from modern software development, can make academia progress faster. To this end I try to implement my research and make it available to others. For example, I developed an R package called freqdom with all results from my PhD.
M: Is your focus on Agile methodology applied to academic research well received by peers or are you facing resistance?
Łukasz: In young and dynamic universities, the fast and risky approach to science is actually encouraged and valued highly. I have the luck to work at the EPFL which is one such university (and one of the best among them).
M: How broadly applicable do you see the Agile methodology within academic research?
Łukasz: In my short academic career I was working on both sides: basic science (mathematical statistics) and applied science (technologies for education). Each side requires different approach, but still fast iterations are necessary in order to succeed. Clearly, the closer the innovation is to applications the more important is the management of the project. Difference between academia and industry is diminishing and thus the techniques for each side are applicable to another.
M: How does this Agile approach fit with traditional academic publication in journals?
Łukasz: There are many ways to bring innovation to the public. For example, one can either publish and promote the ideas in academic community, or build software and promote the research in 'the real world'. Not only the second way allow others to give immediate feedback, but also it simply validates the details of the idea. I believe that github is a better validation of scientific value than any journal peer-review process.
M: Michał, how is your research reaching and impacting the world?
Michał: I turn my research papers into web applications so that anyone immediately can use and test the developed methodology. Currently, I like to use Shiny by RStudio to turn my R code into web apps. Putting the code in action through apps has multiple advantages. From the research perspective it gives your collaborators an easy way to run the developed algorithms and test it on different datasets. Moreover, exposing the developed methodology to the public audience provides a fast feedback. This is priceless and may lead to other interesting areas of research and applications. Finally, such apps are great tools for promoting the research outside of the academia.
M: What is your tool-chain looking like currently?
Łukasz: In my everyday work I use R, python and Matlab, depending on the problem and the application.
Michał: I mainly use R.
M: Have either of you looked at TensorFlow the new release from Google? Any opinions yet?
Yes, we experimented a little bit with TensorFlow in the context of deepart, but apparently, at the current stage, it is hard to outperform the single-GPU implementations. Nevertheless, scalability of TensorFlow will bring fundamental changes and unprecedented opportunities in the machine learning world.
M: If you had substantially greater independent resources what "crazy" idea would you pursue? Why?
Łukasz: I would invest in strong servers and give free access to them to the public. I believe that there is no better research incentive than human curiosity and empowering others can bring much more than the individual efforts.
Michał: I would invest in research and implementation of cool machine learning algorithms that would run on those servers:)
M: What is on your list to learn next?
Łukasz: I would like to learn more about people - to understand them better but also to suggest better suited solutions for existing problems. I am working in a lab which deals with technologies in social science and this has ignited my interest in both sociology and psychology.
Michał: Spark is high on my list.
M: What book(s) do you find yourself turning to repeatedly?
Łukasz: Thinking, Fast and Slow by Daniel Kahneman. It makes me more conscious about my own actions.
Michał: I like this book as well! Antifragile by Nassim Nicholas Taleb is for me another example. It reminds me about the role of randomness and uncertainty in life.
M: Many thanks to Łukasz and Michał for sharing some of their professional pathway and insights.