Kusto: A new query language for OMS Log Analytics
Update!
This blog post is from July 2017 when Microsoft announced that OMS (Operations Management Suite) would be moving to the Kusto platform, and with it the Kusto Query Language (KQL).
Since then – like many things in Azure – there have been some big changes and OMS has transformed into Azure Monitor, and OMS Log Analytics is now Azure Monitor Logs.
So why were we, SquaredUp, writing about OMS and Kusto in 2017?
Back then we were were the first (and perhaps only) dashboard platform to support OMS with our dedicated OMS tile. Since then, we have introduced new tiles for Azure Log Analytics and Azure Application Insights, allowing you to create custom application dashboards that bring your Azure Monitor data together with your existing monitoring from Microsoft System Center Operations Manager (SCOM) into a single pane of glass. Find out more on building dashboards on Azure Monitor with SquaredUp.
Back to the original blog...
Yep, you read that right, there’s a new query language coming to Microsoft’s OMS Log Analytics service!
Hot off the press is the news that there’s going to be a new and significantly enhanced query language and underlying engine for OMS Log Analytics, called Kusto (at least for now). We were in the right place at the right time to see the new platform in action, first-hand, and it looks really impressive.
The MMS session, suitably entitled "A new language for Log Analytics" was hosted by a man often called the 'Godfather of SCOM', Cameron Fuller together with - and we're not sure whether he's called this or not, but he certainly should be - the 'Godfather of SCOM Management Pack authoring', Brian Wren. Any SCOM admin worth his salt will know that those are two of the most respected names in the world of SCOM and that, together with the intriguing title, told us this was a session not to be missed!
And we're pleased to say that the session didn't disappoint! Hats off to Brian, who's an awesome presenter and certainly seemed to be really excited by what he had to share, and little wonder.
Whilst we’ve seen a somewhat lukewarm reaction to OMS from the SCOM customers we speak to, it now looks like Microsoft have something seriously impressive to bring to the table in this area.
Excited by what we’d seen and the possibilities it’s going to open up for customers, we were surprised to find that there’s not much information available about Kusto online yet, at least not in the context of OMS, and so, to help you get started, we’ve pulled together a quick and dirty overview, sharing what we learned at MMS plus what we can figure out from digging through some of the materials that are available online.
Kusto: The low-down
First up, what is it and where’s it come from?
Well, it looks like it’s come from the AppInsights team and is the engine that’s used to power that product. AppInsights was previously part of the Visual Studio (ie. developer-centric) experience, but has recently been moved into OMS which seems like a smart move, bringing code and user-centric monitoring insights together with more traditional operations-focused insights. Hell, it’s what MVP Danielle Grandini’s been asking for since February 2015! Hopefully Daniele’s a happy man now.
Generally speaking, if you dig around for Kusto online, you’ll find that most of the resources currently available come under the AppInsights banner and / or associated with Visual Studio.
One of the most striking things about Kusto overall is how much genuine excitement there seems to be from Microsoft, as this gushing overview demonstrates.
Here are a few tasters why Microsoft seem so excited;
- The ability to run queries across terabytes of data in seconds
- An in-query render functionality which allows users to quickly transform their data into pie charts, time charts, and many other visualizations.
- Compared to an (unnamed) leading competitor in this field, they claim that Kusto is;
- Eight times faster over a 12-hour period
- 100 times faster over a three-day period
- 1000 times faster over a seven-day period
Table inspired by image taken from Brian Harry's blog.
Pause for a moment and re-read that last part.
Between 100 times faster over a three-day period. 1000 times faster over a seven-day period.
Between 100 and 1000 (!) times faster, and not not just anyone, but than an industry-leading competitor. If those figures are even vaguely accurate, that is simply astonishing.
Whilst we can’t comment on this, based on what we saw demo’d at MMS, it certainly looks impressive.
Compared to what’s possible with OMS Log Analytics today, Kusto allows you to create much more complex and sophisticated queries, it makes it a lot easier to construct them and, if the version we saw is anything to go by, delivers a much-improved overall UI experience.
If you're an existing user of Log Analytics in OMS, you'll probably be interested in this new article on upgrading your Azure Log Analytics workspace to new log search.
If you want to know more about integrating OMS and SCOM data within a single UI, and more of our thoughts on the value you can get from Log Analytics generally, check out our recent blog post on the subject.
Lastly, how the heck do you pronounce ‘Kusto’?! Well as far as we can tell, it’s the same as the Cousteau in Jacques Cousteau. Of course, you may not know how to pronounce that either, in which case, this probably isn’t much help!
Update!
A couple of handy updates / new resources have been made available since we first published this blog post;
- Firstly, here's the official announcement from Ketan Ghelani, Group PM for Azure Log Analytics.
- For those of you lusting to get stuck into some technical documentation, you'll find everything you need on the Azure Log Analytics Docs site.
- And for those of you looking to transform your SCOM environment into an Enterprise Application Monitoring machine. Here's a free 30 day trial.
