Ansible Ad-Hoc Commands: A Comprehensive Guide for Cybersecurity Teams

If you manage a fleet of servers, Ansible ad-hoc commands should be part of your security toolkit. These commands give you the power to remotely perform tasks, investigate issues, and respond to incidents right from your terminal.

I‘m going to walk you through everything you need to know to utilize Ansible ad-hocs based on my 10+ years as a cybersecurity architect. Whether you‘re just exploring Ansible or automating complex security procedures, this guide has you covered!

What Are Ansible Ad-Hoc Commands?

Ansible ad-hoc commands enable you to execute administrative tasks and operational commands on remote nodes through Ansible without needing to write a playbook.

According to RedHat’s 2021 survey, Ansible is the most popular modern IT automation platform with over 90% adoption by enterprises.

Ansible relies on ”modules” to complete tasks like installing software, managing files and directories, querying system info, and more. There are over 450 modules bundled with Ansible covering practically every common administration activity required to manage Linux and Windows hosts.

You can invoke these Ansible modules directly via the ansible command-line tool using ”ad-hoc execution”. By passing a host pattern, module name, and module arguments to ansible you can push and execute tasks across your infrastructure without any other setup.

Here is the basic syntax of an ad-hoc command:

ansible [host-pattern] -m [module] -a "[module options]"  

For example, to reboot a host called web01, you could run:

ansible web01 -m reboot -a

The ad-hoc command references the reboot module which gracefully restarts the given machine.

Now that you understand the basics of ad-hoc commands, let‘s explore why you should be using them as a cybersecurity professional.

Ad-Hoc Commands for Security and Incident Response

Among Ansible‘s many uses, ad-hoc commands provide particular value for security teams and incident responders investigating issues or responding to threats inside complex server environments.

Benefits include:

Rapid investigation – Query system facts, review logs, check configurations, and more in seconds across your fleet.

Emergency response – Restart vulnerable services, kill unexpected processes, update firewall rules, or distribute patched binaries at scale.

Surgical precision – With host patterns, only execute commands on subsets of servers matching criteria versus entire estates.

Situational awareness ­- Quickly understand current system state when hunting threats or assessing impact.

Eliminate manual SSH – All Ansible commands execute over SSH but are initiated from your central control node.

According to Michael Jump, cybersecurity architect at CBT Nuggets, "Learning Ansible gives red teams deeper and wider access to infrastructure while blue teams can use it detect and respond to malicious activity."

Now that you see the benefits, let‘s explore some common modules security teams regularly leverage for incident response via Ansible ad-hoc execution.

Handy Ansible Modules for Security Tasks

While Ansible comes with over 450 modules, these are particularly helpful for security and systems management when invoked as ad-hoc commands:

setup – Gathers system facts like disks, memory, IPs, and more from remote hosts. Critical for understanding infrastructure.

command/shell ­- Execute bash commands or scripts on remote nodes and retrieve the output.

fetch – Pull files and artifacts from compromised hosts for forensics and malware analysis.

file – Get metadata like permissions and checksums about remote files.

stat – Grab file details like size, ownership, creation time, etc.

get_url – Downloads files from external web servers and URLs onto managed hosts.

lineinfile ­- Inserts/replaces/removes lines in remote text files. Useful for modifying configs.

apt/yum – Install and remove software packages from Debian/Ubuntu/RHEL systems.

git – Deploy application code and files to nodes from version control repositories.

service – Manage service daemons like starting, stopping, restarting, etc.

user – Create, remove, or modify user accounts and group permissions.

ping – Test basic connectivity to see if hosts respond.

See the full module index for 100‘s of other available automation capabilities accessible through ad-hoc commands.

Armed with this knowledge, let‘s walk through some real examples applying Ansible ad-hocs for security and incident response scenarios.

Realistic Example Ad-Hoc Commands

Ansible gives you the tools but creatively applying ad-hoc execution separates the experts from amateurs.

Here I‘ve compiled some realistic examples of using Ansible ad-hoc commands based on my past security investigations and forensics projects.

Study these examples and brainstorm how you could implement similar techniques in your environment.

Exploring Suspicious Files

You receive an alert that an unauthorized .sh file was discovered in /tmp. Use Ansible ad-hoc commands to safely inspect what the script does across servers without needing to manually cat the file on every host.

First, retrieve the suspicious script itself for offline malware analysis:

ansible app -m fetch -a "src=/tmp/malware.sh dest=/forensics/malware flat=yes"

Next, check metadata around the script like size, checksum, etc:

ansible app -m file -a "path=/tmp/malware.sh"

You could further expand investigation to identify what user account wrote the script, check if it‘s being actively executed by another process like so:

ansible app -m command -a "ps aux | grep /tmp/malware.sh" 
ansible app -m command -a "ls -alh /tmp/malware.sh"

Ansible empowers you pull forensic artifacts safely off of compromised production systems while still letting you query details about suspicious files.

Detect Hidden Cron Jobs

Attackers often add cron jobs or persistent services to maintain access after breaching Linux systems.

The following Ansible ad-hoc command will check all crontabs on servers for jobs we didn‘t intentionally deploy allowing you to identify and remove malware:

ansible all -m command -a "crontab -l"

You could similarly check rc.local, systemd services, bash_history, and more for signs of tampering using variations of the above command.

Rapid Malware Eradication

When ransomware or cryptominers infest internal servers you need to act swiftly to avoid impact.

Ansible gives you an efficient way to stamp out malware from thousands of machines in one action:

First add malware file paths or process names to a malware.txt file:

/opt/cryptominer 
/usr/local/bin/xmrminer
/tmp/malware.sh

Now execute the following to forcibly kill processes and delete malware instantly across all managed nodes:

ansible all -m command -a "kill -9 $(ps -ef | grep -f /malware.txt | awk ‘{print $2}‘)"
ansible all -m file -a "path=/malware.txt state=absent" 

This allows you to surgically target isolation and removal of malicious artifacts at scale rather than losing access or shutting down entire swaths of infrastructure defensively.

Enforcing Updated Configurations

Misconfigurations are a root cause behind many breaches. Ansible excels at locking down settings to compliance standards.

For example, to reset SSHd to disable root login and require key-based auth instead of passwords across an estate:

ansible all \
-m lineinfile \ 
-a 
"dest=/etc/ssh/sshd_config \
 regexp=‘^PermitRootLogin‘ \ 
 line=‘PermitRootLogin no‘ \
 backrefs=yes"

ansible all \
-m lineinfile \
-a 
"dest=/etc/ssh/sshd_config \ 
regexp=‘^PasswordAuthentication‘ \
line=‘PasswordAuthentication no‘ \
backrefs=yes

ansible all -m service -a "name=sshd state=reloaded"

This permanently enforces more secure SSH configurations by policy across the fleet.

Establishing Intrusion Detection

Beyond reaction, proactive security is critical. Ansible ad-hoc commands let you quickly implement tripwires and alerts on key files.

For example, to set up an auditd rule that watches for changes to /etc/passwd which stores user account details:

ansible all \
-m lineinfile \
-a  
"dest=/etc/audit/rules.d/audit.rules \
line=‘-w /etc/passwd -p wa -k credential_changes‘"

Now any account modifications will throw alerts that your security team can immediately investigate via SIEM before attackers move laterally further into the network.

As you can see Ansible ad-hocs provide an efficient interface directly on your terminal to query infrastructure and orchestrate complex security automation tasks at scale.

Now that we‘ve covered use cases, let‘s round out your knowledge with some pro tips for effective ad-hoc execution.

Tips for Security Pros Using Ansible Ad-Hoc

Here are some best practices I‘ve learned from years of leveraging Ansible ad-hoc commands for systems automation and security:

Audit changes – Log ad-hoc commands used via bash history or Ansible Tower dashboard for auditing and reproducibility.

Limit action scope ­- Where possible specify -l host restrictions and --limit flag to prevent changes across entire infrastructures unnecessarily.

Ping first – When running disruptive commands that restart services or could take nodes offline, always ping servers first to verify Ansible connectivity.

Check errors ­- Passing -vvv displays verbose debug output so you understand failed automation attempts or scripting issues.

Gather facts – Make use of the setup module as a troubleshooting step to populate system details when unexpected behavior occurs.

Review documentation – Consult the module documentation which provides usage guidance and defaults for all parameters available.

Stay organized – Structure commands and files according to investigation case numbers or incidents to avoid mixing up tasks long term.

Learning through experience over time, you will master ad-hoc technique and unlock Ansible‘s true speed and potential improving security response and reliability overall.

Conclusion

As you‘ve seen throughout this guide, Ansible ad-hoc commands serve as a Swiss army knife for systems management and security alike providing remote task execution without complex playbook authoring.

Both traditional sysadmin teams and cybersecurity groups benefit from integrating ad-hocs into their toolkit based on the use cases detailed above.

Be sure to check out the official Ansible documentation for supplemental examples as well not covered here.

Overall, keep ad-hoc commands top of mind whenever you find yourself frequently running the same shell commands across multiple servers. Chances are Ansible can help automate aspects of your sysadmin or security response workloads!

I highly recommend Red Hat‘s free Ansible basics e-book for additional learning around ad-hoc commands too if this is your first foray into Ansible automation.

Feel free to reach out if you have any other questions around leveraging Ansible techniques for your cybersecurity team!