In recent years, there has been an increase in attacks, including advanced persistent threats (APTs), and the techniques used by the attacker in these attacks have reached unprecedented sophistication. Threat hunters use various monitoring tools to monitor and collect all these attack actions (which blend in with benign user activities) for cyber threat hunting—the end devices store monitored activities as generated logs/events. Moreover, Organizations like NIST and CIS provide guidelines (CSC) to enforce cyber security and defend against those attacks.
Although the end hosts and networking devices can record all benign user and adversary actions, it is infeasible to monitor everything. In existing approaches, high memory usage and communication overhead to transfer events to the central server create scalability issues on the monitored network. Single event matching on the end-host devices approach to detect attacks generates false alerts, causing the alert fatigue problem. This dissertation presents a distributed hierarchical monitoring agent architecture to overcome those limitations of existing tools and research works.
Additionally, there are no well-defined automated measures and metrics to validate the enforcement of CSC. Manually analyzing and developing measures and metrics to monitor and implementing those monitoring mechanisms are resource-intensive tasks and massively dependent on the security analyst's expertise and knowledge. To tackle those problems, we use LLM as a knowledge base and reasoner to extract measures, metrics, and monitoring mechanism implementation steps from CSC descriptions to reduce the dependency on security analysts with the help of few-shot learning with chain-of-thought prompting. This dissertation presents CSC enforcement assessment with the help of our distributed hierarchical monitoring agent architecture and prompt engineering.