It is expensive. We forward logs to spkunk (we run our own instances). Splunk is really solid. All the logs are json and require certain fields. We use it for tend analysis, alerting, graphs, reports, and digging into production issues. It digs through terabytes of data relatively quickly.
I think Splunk is the reason I can’t recommend ELK, Splunk simply makes ELK look almost non-functional. This is years ago but we tried Splunk for a year, but the switched to ELK due to pricing. Our number of searches dropped to almost zero after that, because usability was so poor, in comparison.
As a result we didn’t utilize the data we had, or in many cases reverted to using grep.
If you want a cheaper alternative, Humio has become rather good and is relatively easy to use.
