An Unexpected Journey...

2012 was a busy year that passed by too fast and without any blog entries. It was also a year, where I've switched sides from being a customer into being a consultant.

I was running the largest e-mail platform in Switzerland for over three years, together with an outstanding team dedicated to the task, but somewhere deep in the guts, I had this feeling to set sails...

It was the Splunk .conf in 2011, that was the impulse that sent me to an unexpected journey. I did a presentation there, and got into a lot of discussions with other Splunk users. Something was going on, Splunk has moved from niche into something people were enthusiastic about. I felt like in 2007 when Splunk 3.0 came out, that something was brewing...

Meno, the second Splunk customer in Switzerland, had already taken the path of working as a consultant at a Splunk Partner. I knew that they had an open position, but I declined the job a couple of months before, as I was satisfied with my current position.

I mean, where else in Switzerland can you run the full HW/Storage/Software-Stack in a large environment? I learned so much in this environment, and I was afraid, that I would not find something similarly interesting and challenging.

Then the year 2012 came. I started in January, and after only 3 weeks I was already booked out by customers who wanted to start Splunk projects.

What fascinates me, is the diversity of the projects. They go through all industries, can be highly technical and the next day very business oriented. I have to talk to all kinds of people, security, auditors, server and storage admins, network engineers, developers... You name it. One thing Splunk projects have in common: you never know what to expect at the customer, but in the end you have a happy customer.

Splunk .conf2012 was even more exciting than the previous year. Again, I was doing a presentation, and this time the room was overbooked! I was chatting with many Splunk employees, customers, other partners. Everyone was enthusiastic about the products and things to come.
 


If we look at the earlier years, many customers did not know anything about unstructured / machine data. In 2012, this clearly changed. People learned about Splunk from various Splunk Live! events, roadshows, and mouth-to-mouth propaganda. Customers really wanted to have Splunk. The only problem they had was how to start with such a powerful tool. Often projects started as proof-of-concepts, to give customers more ideas about what to do with the data. After implementing a few use-cases customers grasped the idea behind Splunk and more and more ideas popped up.

2013 is already here and I guess, it will continue the same way for new customers, learning what value Splunk can bring. Customers who have used Splunk for a longer time, will try to get even more value out of their machine data/unstructured data. I see some growing interest in data visualizations,  machine learning and predictive analytics. Also, Splunk is moving up the stack more and more, from giving technology answers, into giving business answers.

I expect this year to be even more exiting than last year, not only for me, but also the company I work for. There will be quite some announcements this year... stay tuned!

Comments

Post a Comment

Popular posts from this blog

Heating up the Data Pipeline (Part 1)

Image
Pre-Processing Data I often hear the question from our customers, how data can be transformed prior to indexing in Splunk. Damien from  Baboonbones  has done a tremendous job in creating add-ons   providing custom inputs for Splunk. Most of his custom inputs provide the means to pre-process data by allowing custom event handlers to be written. Sometimes you still want to pre-process data that gets collected from Splunk's standard input types, like file monitors, Windows EventLogs, scripted inputs etc. Also, not everyone is capable of writing custom event handlers. A requirement these customers have, is that they have rolled out a large number of Splunk Universal Forwarders and they do not want to install another agent. To summarize, the solution capable of pre-processing data, should be easy to use, be easily integrated and be build on top of their existing architecture. How to plumb Splunk Pipelines Splunk has its own fittings to connect a Univer...
Read more

Heating up the Data Pipeline (Part 3)

Image
Welcome back to the "Heating up the Data Pipeline" blog series. In part 1 we talked about how to route data from Splunk to a 3rd party system. In part 2 walked through a simple data flow that passes data collected from Splunk Forwarders through Apache NiFi back to Splunk over the HTTP Event Collector. In this part, we will look at a more complex use case, where we route events to an index, based on the sending host's classification. The classification will be looked up from a CSV file. We will make use of Apache NiFi's new Record-Oriented  data handling capabilities, which will look initially a bit more complicated, but once you grasp it, it will make further Use Cases easier and faster to build. High-Level Dataflow We will again start with our ListentTCP input, but this time we will send the data to another Processor Group. The Processor Group will again emit events suitable for Splunk HEC. Note that we have increased the Max Batch Size from 1 ...
Read more

Heating up the Data Pipeline (Part 2)

Image
In Part 1 we went through how to route events from Splunk to a 3rd party system without losing metadata. Now I'll show you how events can be transformed using Apache NiFi and be sent back to Splunk into the  HTTP Event Collector . Note: The following is not a step-by-step documentation. To learn how to use Apache NiFi you should read the G etting Started Guide . Simple Pass-Through Flow As a first exercise we will create a simple flow, that only passes data through NiFi, without applying any complex transformations. The following picture shows a high-level NiFi flow, that receives events in our custom uncooked event format with a TCP listener, then sends the data further into a transformation "black box" (aka Processor Group), which emits events in a format, that can be ingested into a Splunk HTTP Event Collector input. Apache Nifi currently provides a rapidly growing number of processors (currently 266), which can be used for data ingestion, transfo...
Read more