Tag: splunk

Enable Real-time Search in Splunk that expires automatically.

Here I wrote a script that allows you to enable real-time search for a role and set a timer for it to disable again automatically. I came up with the idea because we wanted the ability for people to use real-time search, but without the fear of them abusing it. I used a Splunk API and a little Bash Shell scripting to make this work. It works like this, you kick off the script on the SH passing it the role and the time (in minutes) that you want it to be enabled for. The script uses an API call to add the rtsearch capability to the role and the removes it again when the timer expires. I designed this script so that it will collect the current role capabilities before anything is changed  so that after the timer expires it can restore the capabilities back to what they were without rtsearch. This script could easily be converted to accept any capability as another argument and allow you to add anything you want other than rtsearch.

Script after the break..

Read more

Create indexes for Splunk automatically.

Our Splunk environment uses nearly a thousand indexes per region or cluster with many more being added daily. Why so many indexes, well it’s all about administration. Let us say you have 5 separate websites or apps and each of the sites are managed by a separate team. Maybe each of these sites has a test and production instance, so for each site there is an index for test and an index for production. This means we have 10 indexes now for these 5 sites and those indexes can be specifically assigned to individual teams. Now imagine our environment with hundreds of sites with different environments and most of these managed by different teams. That is how we have such a large number of indexes with more being added every day. So the standard process of adding indexes just doesn’t make sense for our needs. So, I created some bash scripts that are run by cron jobs to automate the process.

Read more