Tag: Quandl

Tammer Kamel is CEO and co-founder of Quandl

What is Quandl and what was the inspiration for creating it?

Quandl is a service that delivers financial data.  We bring together over 25 million financial datasets on a single website, and make this data available to analysts in any form they want.

The inspiration for Quandl was my own frustration working with data when I was an analyst. I’ll give you one example among many: I was working in Python trying to create a simulation around oil and uranium prices for the past 20 years. It was a struggle to find the datasets on Google. When I finally did, the formats were a mess. It took half an hour to get them formatted and merged. Then I had to repeat the same exercise every day to update the model.

This lead me to think, “Why isn’t there a platform where I can type in ‘uranium prices’ and get a clean dataset? Why can’t I get it directly into Python, or other tools I use, like Excel, R and Matlab?”. I knew that a platform that could take a tedious, half-hour process and cut it down to 10 seconds would make life easier for me and probably millions of other people. That was the inspiration.

Why has it been hard historically for everyone to access financial data? Analysts had it but the rest of us didn’t. Why?

Data used to be a scarce resource: hard to produce, hard to acquire, hard to use effectively. As a result, only the largest institutions could afford to pay for data. Individuals and smaller firms were priced out.

But that era is gone. Today, we live in an age of data ubiquity: there’s data everywhere. Anyone with a web connection can get stock quotes, or currency exchange rates, or company financials, or demographic forecasts. Access to raw data is no longer a limiting factor. That’s a huge contributor to the rise of solutions like Quandl.

Of course, raw data can only take you so far. For data to be usable, it needs to be cleaned, structured, documented, and quality-controlled. This is where the democratizing power of the internet comes into play. Quandl’s marketplace model replaces inefficient, pre-internet, “factory-style” data production with a network of specialist data creators and vendors.

Quandl gives these publishers a distribution channel and a transaction mechanism. It lets them make their databases available to the whole world. Thanks to Quandl, they can start competing with larger publishers. And they’ve been amazingly effective at that, as our success shows.

A second dynamic at work is unbundling. It’s similar to what is happening to cable TV. With traditional terminals, users were forced to pay an annual subscription to every database, whether they needed it or not. It was the business model, just like cable’s business model forced consumers to buy a bundle of 500 channels when they only watched an average of 15. With Quandl, users only pay for the databases they need so we’re much more accessible.

Both of these trends – scarcity replaced by abundance, and bundles replaced by choice – are patterns we’ve seen play out time and again in other industries. Internet platforms tend to disrupt pre-internet businesses. We’re seeing it happen now in the world of financial data.

Looking at Product Hunt, it appears a lot of new apps are being built using your APIs. What’s happening here and what does this ecosystem look like years down the road?

Yes, this is an exciting development for us. Currently, there are over a hundred financial and analytical apps built on top of Quandl’s API, all created by the Quandl community.

You can get Quandl data into scientific and mathematical tools like Mathematica, Matlab, Maple, Octave, R, SAS and Stata. You can also access our data from general purpose languages like C/C++/C#, Java, Julia, Python and Ruby. There are integrations for trading platforms like AmiBroker, Money.Net, Quantopian, Tiingo and TradingView; and for analytic tools like Mode, Plotly and Statwing. Finally, there are dozens of business intelligence and financial apps with sophisticated, next generation analytics – like Ayasdi, Domo, Kensho and Premise – that use our data.

Our philosophy here is quite simple: “Let a thousand flowers bloom.” Instead of getting into analytics or visualization ourselves, we empower other businesses to build their own solutions. We’ve seen other data providers restrict users to the analytics already present in their terminals. This means that users can’t choose the tool they want to work in.

It also means that they’re paying for all the analytics, just like they’re paying for all the databases, even if they only use a fraction of them. It’s the 500 channel problem all over again – they have to pay for a bundle of analytics, everything from yield curve calculators to option pricers to commodity shipping analysis, just to get the few that they actually need.

If you’re a bond trader, you don’t need an equity valuation model. If you’re a commodity trader, you don’t need a yield curve calculator.  If you’re a market technician, you don’t need a fundamentals-based screener. But if you’re a terminal subscriber, you’re paying for everything.

The app model, on the other hand, allows each analyst to select their own tools. Not only is it more economic, it’s also more powerful: it allows users to pick and choose the precise apps that are optimal for their needs, instead of a generic one-size-fits-all approach.

Our vision for a few years down the road is a rich ecosystem of apps powered by Quandl.  Each app is hyper-focused and tailor-made for each specific use case: a tool for every task, and every task with its own tool. We believe this is the future of data analysis.

Are there particular challenges you’ve encountered in building out Quandl — how did you overcome them?

Building a delightful user experience is always a challenge. There are so many ways that people use data. We want to give our users maximum flexibility, but we also want to keep it simple. We think we’ve achieved this balance well so far, but there’s always a temptation to add more complexity. We’ve put a lot of resources into making the experience faster, easier and more intuitive. This hasn’t been a priority in the data industry, but it’s a priority for us.

While the front end might appear simple, the back end is anything but.  We’ve had to solve some hard technical challenges to build Quandl.  Bringing together millions of datasets from thousands of different publishers on a single, unified platform requires non-trivial advances in data parsing, structuring and delivery.

Another challenge is scale: building infrastructure to serve 50 million data downloads a month with a millisecond response time. Our clients include many of the biggest banks, asset managers and hedge funds in the world, so speed and reliability are crucial. We’ve done some exceptional engineering around the Quandl API to make this happen.

What’s next for 2016?

Our highest priority, for 2016 and probably forever, is adding new data to the platform. The more data we have, the better we become for all our users.

In addition to expanding our time-series data coverage, we will be adding a couple of new data types to Quandl: non-time-series data and intraday data.  We’ll also be adding more vendors and unique databases.

The Quandl website, ecosystem and feature set will continue to evolve as we listen to what our customers need. That’s a never-ending process of improvement for us.

Photo credit: Glyn Lowe Photoworks. via Visual hunt / CC BY