Create 3 node Hadoop and Spark Cluster Monitored by SignalFx from Splunk

George Jen, Jen Tek LLC

Summary

This writing provides step by step instructions to create 3 node Hadoop, Hive and 3 node Spark cluster in both Standalone mode and managed by yarn from Hadoop as well as how to setup monitoring by SignalFx from Splunk using VirtualBox CentOS VMs on a single Windows 10 hosting PC. Hope it is helpful.

Objective and Task

I need to test a monitoring solution on Hadoop and Spark installation, to monitor key Hadoop cluster and Spark…