Python – Page 2 – Baker Street Forensics

Beyond Hashes: Simplifying Malware Identification with Python and MpCmdRun

DFIR, Forensics, Malware, PowerShell, Python

In an earlier post titled “Growing Your Malware Corpus”, I outlined methods for building a comprehensive test corpus of malware for detection engineering. It covers using sources like VX-Underground for malware samples and details how to organize and unzip these files using Python scripts.

In today’s post we’re going to cover using Python to apply a standard naming methodology to all our malware samples.

Depending on where you curate your samples from, they could be named by their hash, or as they were identified during investigation, like invoice.exe. Depending on the size of your collection, I’d surmise it’s highly unlikely that they have a consistent naming format.

I don’t know about you, but a title that indicates the malware family and platform is a lot more useful to me than a hash value when perusing the corpus for a juicy malware sample. We can rename all our malware files using Python and the command line utility for Windows Defender.

Step 1: You’ll need to install Python on a Windows box that has Windows Defender.

Install Python

If you don’t have Python installed on your Windows machine, you can do so by downloading the installer from python.org, or alternatively, installing from the Windows store.

*Windows Store installer for Python versions 3.7 to 3.12*

Directory Exclusion

Within the Windows Defender Virus & Threat protection settings, add an exclusion for the directory you’re going to be using with the malware. Make sure the exclusion is in place before connecting the drive with the malware so it doesn’t get nuked.

Note: Doing this assumes you’ve evaluated the potential risks associated with handling malware, even in controlled settings, and have taken safety precautions. This is not an exercise to be conducted on your corporate workstation.

*Screenshot of the D:\Malware Directory being excluded from Windows Defender.*

Automatic Sample submission

It’s up to you if you want to disable the Automatic Sample submission. If you do, you’ll still may get prompted to send some.

*Windows Defender requesting to send samples to Microsoft for further analysis.*

Rename_Malware.py

The star of this show is the python script that was shared on ~~twitter~~ from vx-underground.

The post walks through various options for utilizing Windows Defender command line, MpCPmdRun.exe. Using that information a Python script was developed to loop through the contents of a directory, analyze those files with Windows Defender, and then rename the files accordingly based on the malware identification.

*Python code for rename_malware.py in VS Code.*

You can grab the code from the linked post, or a copy on my Github here.

Once you’ve got Python installed, directory exclusion configured, and a pocketful of kryptonite (malware), – you’re ready to go.

python rename_malware.py D:\Malware

Windows Defender command line will run through each file and rename them based on its detection.

*The script recursively renames the analyzed files.*

I’m running this on a copy of my malware corpus of 30,000+ malware samples.

Counting the Corpus

A bit of handy PowerShell math. Before and after the process I wanted to be sure of how many files were present to ensure that the antivirus didn’t remove any. I also wanted to exclude counting pdfs as many of the samples in my corpus also have accompanying write-ups.

*Using PowerShell for selective file counting.*

Get-ChildItem -Recurse -file | Where-Object { $_.Extension ne *.pdf" } | Measure-Object | Select Count

Back at the console the script is still running.

*The script continues recursively renaming the analyzed files.*

Finally… not begrudgingly at all considering over 30,000 samples were analyzed, the script has reached the end of the samples.

*Script has reached the end of the files.*

If we do a directory listing on the contents of the malware directory, we see that the majority of the files have all been renamed based on their malware identification.

*File listing showing malware files named Trojan.Powershell… Trojan.Script… etc.*

Hooray!

That makes it much easier to search and query through the malware repository.

The last step… make a BACKUP. 😉

Growing Your Malware Corpus

DFIR, Malware, Python, yara

If you’re writing YARA rules or doing other kinds of detection engineering, you’ll want to have a test bed that you can run your rules against. This is known as a corpus. For your corpus you’ll want to have both Goodware (known good operating system files), as well as a library of malware files.

One source to get a lot of malware samples is from VX-Underground. What I really appreciate about VX-Underground is that in addition to providing lots of malware samples, they also produce an annual archive of samples and papers. You can download a whole year’s worth of samples and papers, from 2010 to 2023.

Pandora’s Box

Just to understand the structure here, I have a USB device called “Pandora.” On the root of the drive is a folder called “APT”, and within that is a “Samples” directory. Inside the samples directory is the .7z download for 2023 from VX-Underground. There’s also a python script… we’ll get to that soon enough.

The first thing we’ll need to do is unzip the download with the usual password.

7zz x 2023.7z

Once the initial extraction is complete you can delete the original 2023.7z archive.

Within the archive for each year, there is a directory for the sample, with sub-directories of ‘Samples’ and ‘Papers.’ Every one of the samples is also password protected zip file.

This makes sense from a safety perspective, but it makes it impossible to scan against all the files at once.

Python to the Rescue

We can utilize a Python script to recursively go through the contents of our malware folder and unzip all the password protected files, while keeping those files in their original directories.

You may have noticed in the first screenshot that I have a script called ExtractSamples.py in my APT directory.

We will use this for the recursive password protected extractions.

Python ExtractSamples.py

A flurry of code goes by, and you congratulate yourself on you Python prowess. Now if we look again at our contents, we’ve got the extracted sample and the original zip file.

Let’s get rid of all the zip files as we don’t need them cluttering up the corpus.

We can start by running a find command to identify all the 7zip files.

find . -type f -name '*.7z' -print

After you’ve checked the output and verified the command above is only grabbing the 7z files you want to delete, we can update the command to delete the found files.

find . -type f -name '*.7z' -delete

One more a directory listing to verify:

Success. All the 7z files are removed and all the sample files are intact.

GitHub Link: ExtractSamples.py

Time to go write some new detections!

Huntress CTF: Week 1 – Forensics: Backdoored Splunk, Traffic, Dumpster Fire

CTF, DFIR, PowerShell, Python

Backdoored Splunk

Hit Start.

So we’ve got a url and a specific port. Firefox web browser yields…

So we need an Authorization header. 🤔

Time to look at the provided files. It looks to be the export of a Splunk application.

Time to download an eval copy of Splunk and… pause. There’s probably a simpler way to attack this.

The Silver Searcher is a command line tool I picked up during the CTF and I love it. It’s like Grep on PCP.

Once installed, the base command is ag, followed by what you’re searching for, and where. So let’s do a quick search for Authorization on all the contents of this directory.

That looks interesting. A clue? One of the PowerShell scripts has Authorization and what looks to be Base64 code.

We also see a comment about the $PORT being dynamic based on the Start button. Decoding the string in CyberChef…

At this point we have all the pieces, we just need to put them together. I started to look at different ways to pass an Authorization header to a web server. There’s proxy tools galore. And then there’s the basic’s like curl. After a bit of brushing up on my syntax I had:

curl -H "Authorization: Basic [longStringFromThePowershell]" http://site:$PORT

Yay what looks like more Base64. Once more with our Chef’s hat and…

Traffic

rita was a tool I hadn’t used before but it was very easy to use. I installed it on my REMnux box and then ran it against the dataset.

I then used the command to generate an html report.

Looking through the DNS requests there’s something sketchy indeed.

Let’s go take a look at that.

Dumpster Fire

Let’s start with the_silver_searcher again and see if we have any luck with “Password”.

There’s a number of hits including references to an encryptedUsername and encryptedPassword in the logins.json file. So we’ve got some encrypted Firefox user passwords. If only there were a utility that could decrypt those. Enter firepwd.py, an open source tool to decrypt Mozilla protected passwords.

Run the script in Python and point it to the directory for the user profile (where the logins.json file is).

That’s a pretty LEET password 😉

Use the tag #HuntressCTF on BakerStreetForensics.com to see all related posts and solutions for the 2023 Huntress CTF.

Creating YARA files with Python

DFIR, Malware, Python, yara

When I’m researching a piece of malware, I’ll have a notepad open (usually VS Code), where I’m capturing strings that might be useful for a detection rule. When I have a good set of indicators, the next step is to turn them into a YARA rule.

It’s easy enough to create a YARA file by hand. My objective was to streamline the boring stuff like formatting and generating a string identifier ($s1 = “stringOne”) for each string. Normally PowerShell is my goto, but this week I’m branching out and wanted to work on my Python coding.

The code relies on you having a file called strings.txt. One string per line.

When you run the script it will prompt for (metadata):

rule name
author
description
hash

It then takes the contents of strings.txt and combines those with the metadata to produce a cleanly formatted YARA rule.

Caveats:

If the strings have special characters that need to be escaped, you may need to tweak the strings in the rule after it’s created.

The script will define the condition “any of them”. If you prefer to have all strings required, you can change line 22 from

yara_rule += '\t\tany of them\n}\n'

yara_rule += '\t\tall of them\n}\n'

CreateYARA.py

def get_user_input():
    rule_name = input("Enter the rule name: ")
    author = input("Enter the author: ")
    description = input("Enter the description: ")
    hash_value = input("Enter the hash value: ")
    return rule_name, author, description, hash_value

def create_yara_rule(rule_name, author, description, hash_value, strings_file):
    yara_rule = f'''rule {rule_name} {{
    meta:
    \tauthor = "{author}"
    \tdescription = "{description}"
    \thash = "{hash_value}"

    strings:
    '''
    with open(strings_file, 'r') as file:
        for id, line in enumerate(file, start=1):
            yara_rule += f'\t$s{id} = "{line.strip()}"\n\t'
    yara_rule += '\n'
    yara_rule += '\tcondition:\n'
    yara_rule += '\t\tany of them\n}\n'

    return yara_rule

def main():
    rule_name, author, description, hash_value = get_user_input()
    strings_file = 'strings.txt'  

    yara_rule = create_yara_rule(rule_name, author, description, hash_value, strings_file)
    print("Generated YARA rule:")
    print(yara_rule)
    
    yar_filename = f'{rule_name}.yar'
    with open(yar_filename, 'w') as yar_file:
        yar_file.write(yara_rule)

    print(f"YARA rule saved to {yar_filename}")

if __name__ == "__main__":
    main()