KAPE batch mode, ARM Memory, updates to CSIRT-Collect, and all the things I learned along the way.

A couple weeks ago, a reader commented on the post Adding RAM collections to KAPE triage, “Couldn’t this be implemented by using linear processing with KAPE in batch mode?” and I couldn’t be more grateful for their inquiry.

When I was first introduced to KAPE, ‘batch mode’ didn’t exist yet as a feature, so it was something I had to get familiar with. From the documentation:

“Batch mode works by placing one or more command lines (without the kape.exe part) into a file named _kape.cli. This file should contain one full command line on each line. This allows you to preconfigure KAPE to perform a given action (for example, collect certain files, zip them, then SFTP them to somewhere).”

Essentially it allows you to string together multiple KAPE jobs and run them together. This could be useful when you want to send the output of one command to a network share, and another to S3. I’m sure there are many other use cases, but I haven’t explored any as of yet.

KAPE has supported memory collection from its very early days. In fact I wrote the original DumpIt plugin for KAPE back in May ’19 (Pepperidge Farm Remembers). I had the privilege of attending the first Beta class of SANS FOR498 with Eric Zimmerman and Kevin Ripa when KAPE was first coming on to the scene. What a duo to learn from. I later wrote the module for Magnet RAM Capture. Rounding out the triad is Winpmem which was added by another contributor around the same time (hours apart if I remember) as the DumpIt plugin.

Preserving the most volatile data first

At the time I thought I’d finally solved my Incident Responders ultimate dream of ‘give me RAM and a selective triage image – and give it to me quickly.’ I was pretty giddy over having made what would be my first public contributions to DFIR software. Using any of the memory modules with KAPE, and an appropriate Targets selection for triage would yield both results, but to me there was still a problem. The way KAPE operates it initiates the Targets operations and then the Modules operations. If I selected a triage image and included the memory collection module, the triage collection would run first and then the memory collection. I really wanted to grab the memory first and then do any other operations, with the goal of preserving as much volatile data as possible. It was out of that need to control the order of operations that led me to write CSIRT-Collect.

Via PowerShell, CSIRT-Collect would collect a RAM image first (evolving over the years experimenting with DumpIt, Winpmen, and Magnet RAM Capture – and now back to DumpIt again), and then invoke KAPE to handle the triage collection. There were two versions of the code, mostly identical but adapted for use via network share or via USB drive. We used this script within my IR team for a few years before I shared it to GitHub where it’s continued to develop (and be the topic of numerous webcasts.)

Renaming the Project

Getting the ASCII art to match the BakerStreetForensics logo was a labor of love, let me tell you. It was actually that update that led to the thought to rename the project. While CSIRT-Collect was somewhat indicative of what the scripts function was, it wasn’t especially memorable. The initial naming was based on its internal use within our IR team.

As of the newest release, following an official vote on Mastodon ;), the project and repo have been renamed from CSIRT-Collect to CyberPipe. What can I say I’m a fan of puns. Besides the not-so subtle Sherlockian reference, there’s also the function of pipe (or | ), a method used to pass information from one program process to another, which is just what CyberPipe does.

OK, back to the technical details.

ARM64 processors

DumpIt, which was recently added to the Magnet Forensics Free Tools site, supports x64, x86 and ARM processors on Windows. With CSIRT-Collect and KAPE, I was able to run the triage and RAM collections on my regular suspects Windows instances, as well as a Windows ARM virtual machine running on an M1 Mac.

That introduced my next problem though, that there wasn’t a means within KAPE to detect ARM64 vs x64, just 32-bit vs. 64-bit. Since the architecture support wasn’t there, I whipped up a new KAPE module specific to the ARM version of DumpIt which has been added to the latest version of KAPE.


Once you update KAPE, grab the ARM version of DumpIt, name it DumpIt_arm.exe, drop it in the /bin and you’re good to go. CyberPipe queries the system and will direct KAPE to utilize the appropriate .exe for the architecture.

CyberPipe will also run on 32-bit instances. In this case you’ll need the 32-bit version of DumpIt.exe in /modules/bin as DumpIt_x86.exe. In this case KAPE can make the determination of 32-bit vs 64, so no changes are needed for the PowerShell script. (See the 64-bit: False in screenshot below).

Back to batch mode

The first line of the _kape.cli had the (modules) arguments for DumpIt and Encrypted Disk Detector.
The next line called the KAPE-Triage (targets) collection.

--msource C:\ --mdest $dest --module DumpIt_Memory,MagnetForensics_EDD 
--tsource C:\ --tdest $dest --target KapeTriage --vhdx $env:computername --zv false

By default all the KAPE instances called by the _kape.cli will execute simultaneously. When I was first experimenting with it both the RAM collection and the triage collection would initiate at the same time. While this made for even faster collections on larger targets, there was still the issue of preserving the most volatile data. Back to the documentation, (it helps to read to the very last line… ) -thanks, Brian.

“Should you want to limit things to a single KAPE process running at once, adding --ul to one of the entries (it should be the first one in most cases) tells KAPE to wait for each instance of KAPE to exit before starting the next. When using this mode, you will only see one active instance of KAPE vs. multiple instances starting at once.”

Adding a –ul to the first KAPE argument ensured that the memory collection operations would take place first and only then when completed start the next phase of the triage collection.

--msource C:\ --mdest $dest --module DumpIt_Memory,MagnetForensics_EDD --ul
--tsource C:\ --tdest $dest --target KapeTriage --vhdx $env:computername --zv false

Another bit about batch mode. When kape.exe executes and there is a _kape.cli in the root path it will use those arguments and then rename the file to $timestamp_kape.cli. So the next time KAPE executes from this directory, a _kape.cli will not be present. Since the intention of the script is to be reusable and repeatable without intervention, I needed the _kape.cli to be persistent. I decided the solution was to have the script create the _kape.cli at runtime. This also allowed me to accommodate for the x64 vs ARM64 issue. If ARM is detected, the _kape.cli will be generated with the instructions for the ARM module. Otherwise the x64 module will be used by default.

$arm = (Get-WmiObject -Class Win32_ComputerSystem).SystemType -match '(ARM)'
if ($arm -eq "True") {
     Write-Host "ARM detected"
     Set-Content -Path _kape.cli -Value "--msource C:\ --mdest $dest --module DumpIt_Memory_ARM,MagnetForensics_EDD --ul" }
else {
    Set-Content -Path _kape.cli -Value "--msource C:\ --mdest $dest --module DumpIt_Memory,MagnetForensics_EDD --ul" }
Add-Content -Path _kape.cli -Value "--tsource C:\ --tdest $dest --target KapeTriage --vhdx $env:computername --zv false"

Other enhancements

Up until recently the script required specific folder configurations in place (Collections folder, Memory folder, KAPE, etc.) That’s been simplified now. Just sit CyberPipe.ps1 next to your KAPE directory (whether on network or USB) and the script will take care of any folder creation necessary.

v4.0 Features:

– “One Script to Rule them All”
– Admin permissions check before execution

param ([switch]$Elevated)
function Test-Admin {
    $currentUser = New-Object Security.Principal.WindowsPrincipal $([Security.Principal.WindowsIdentity]::GetCurrent())
    $currentUser.IsInRole([Security.Principal.WindowsBuiltinRole]::Administrator)
}
if ((Test-Admin) -eq $false)  {
    if ($elevated) {
    } else {
        Write-host -fore DarkYellow "CyberPipe requires Admin permissions (not detected). Exiting."        
    }
    exit
}

– Memory acquisition will use Magnet DumpIt for Windows (previously used Magnet RAM Capture).
– Support for x64, ARM64 and x86 architectures.
– Both memory acquistion and triage collection now facilitated via KAPE batch mode with _kape.cli dynamically built during execution.
– Capture directories now named to $hostname-$timestamp to support multiple collections from the same asset without overwriting.

$collection = $env:COMPUTERNAME+$tstamp

– Alert if Bitlocker key not detected. Both display and (empty) text file updated if encryption key not detected.
– If key is detected it is written to the output file.

(Get-BitLockerVolume -MountPoint C).KeyProtector > $CollectionHostpath\LiveResponse\$collection-key.txt 
If ($Null -eq (Get-Content "$CollectionHostpath\LiveResponse\$collection-key.txt")) {
Write-Host -Fore Yellow "Bitlocker key not identified."
Set-Content -Path $CollectionHostpath\LiveResponse\$collection-key.txt -Value "No Bitlocker key identified for $env:computername"
}
Else {
    Write-Host -fore green "Bitlocker key recovered."
}

– More efficient use of variables for output files rather than relying on renaming functions during operations.
– Now just one script for Network or USB usage. Uncomment the “Network Collection” section for network use.

## Network Collection - uncomment the section below for Network use
<#
Write-Host -Fore Gray "Mapping network drive..."
$Networkpath = "X:\" 
If (Test-Path -Path $Networkpath) {
    Write-Host -Fore Gray "Drive Exists already."
}
Else {
    # map network drive
    (New-Object -ComObject WScript.Network).MapNetworkDrive("X:","\\Server\Triage")
    # check mapping again
    If (Test-Path -Path $Networkpath) {
        Write-Host -Fore Gray "Drive has been mapped."
    }
    Else {
        Write-Host -Fore Red "Error mapping drive."
    }
}
Set-Location X:
#>
## Below is for USB and Network:
$tstamp = (Get-Date -Format "_yyyyMMddHHmm")
$collection = $env:COMPUTERNAME+$tstamp
...

– Stopwatch function will calculate the total runtime of the collection.
– ASCII art 😉 “Ceci n’est pas une pipe.”

Examples of CyberPipe in Action

CyberPipe collection of ARM64 asset to HD.
CyberPipe collection of x32 asset to HD.
CyberPipe collection of x64 asset to Network share.
CyberPipe collection of x64 asset to SSD.

I’m very pleased with how this project continues to develop. Special thanks to Brian Maloney whos question got things started down a new path. If you have other questions or suggestions, feel free to leave them here – or better yet, add the conversation on Github.

On February 21, I’ll be discussing CSIRT-Collect and Free Tools to Bolster Your IR Toolkit at the Magnet Virtual Summit #MVS2023. Register today!

In April you can catch me in Nashville at the Magnet User Summit where I’ll be discussing Magnet2Go. Building A ‘Windows To Go’ Drive To Support Offline Collections, which will also feature the new CyberPipe script.

Can’t wait for the conferences? Head over to GitHub and grab your copy of CyberPipe today.

Group collections from O365 with PowerShell

If you’re working in or responding to an O365 environment, there’s plenty of opportunities where you need to search and collect from multiple O365 custodians at the same time. While the experience of the Security & Compliance Center has improved over the years, I still find it inefficient for creating larger collections – especially when each custodian has to be searched for and added one at a time.

I created a handful of PowerShell scripts that automate the creation of searches for a group of custodians (provided via .txt file). I’ve used these methods countless times for both eDiscovery and IR cases.

There are different scripts to address the collection of:

  • O365 Mailbox – will capture email, calendar, tasks, contacts, MS Teams*.
  • Microsoft Teams – either for a single custodian or for a group.
  • Microsoft OneDrive – collect the O365 OneDrive for Business for a group of custodians.
  • When Legal says “get it all” – All O365 mailbox contents, including Teams, and OneDrive.

Once the collection has been generated you will still need to log on to https://protection.office.com to retrieve the search results.

Prerequisites:

  • ExchaneOnlineManagment PowerShell Module
  • Microsoft.Online.SharePoint.PowerShell Module

O365 Mailbox Collections: MSExchangeGroupSearch.ps1

<# MS Exchange Security & Compliance Search 
version 2.0
https://github.com/dwmetz/Axiom-PowerShell
Author: @dwmetz
Function:
    Collect an O365 mailbox search for group of custodians.
    Note this script requires previous installation of the ExchangeOnlineManagement PowerShell module
    See https://docs.microsoft.com/en-us/powershell/exchange/connect-to-scc-powershell?view=exchange-ps for more information.
      
This PowerShell script will prompt you for the following information:
    * Your user credentials                                          
    * The pathname for the text file that contains a list of user email addresses
    * The name of the Content Search that will be created
    * The search query string
The script will then:
    * Create and start a Content Search using the above information

Updates:
    17.November.2022 - updated ExchangeOnlineManagement connection, Security & Compliance Center (IPPSSession)

#>
# New Auth
Import-module ExchangeOnlineManagement
Connect-IPPSSession
# Get other required information
$inputfile = read-host "Enter the file name of the text file that contains the email addresses for the users you want to search"
$searchName = Read-Host "Enter the name for the new search"
$searchQuery = Read-Host "[Optional] Enter the search query you want to use"
$emailAddresses = Get-Content $inputfile | Where-Object {$_ -ne ""}  | ForEach-Object{ $_.Trim() }
Write-Host "Creating and starting the search"
$search = New-ComplianceSearch -Name $searchName -ExchangeLocation $emailAddresses -ContentMatchQuery $searchQuery
# Finally, start the search and then display the status
if($search)
{
    Start-ComplianceSearch $search.Name
    Get-ComplianceSearch $search.Name
} 
Write-Host "Search initiated"-ForegroundColor Blue
Write-Host "Proceed to https://protection.office.com/ to download the results."-ForegroundColor Blue

O365 Mailboxes and OneDrives: MS-ExchangeODGroupSearch.ps1

Note: you will get 2 authentication prompts as you are logging on to Security & Compliance Center as well as the Sharepoint Admin panel.

<# MS Exchange & OneDrive Security & Compliance Search 
version 2.0
https://github.com/dwmetz/Axiom-PowerShell
Author: @dwmetz

Function: This script will generate a Security and Compliance Search to capture O365 Email and OneDrive for a list of custodians.

This PowerShell script will prompt you for the following information:
    * Your user credentials                                          
    * The pathname for the text file that contains a list of user email addresses
    * The name of the Content Search that will be created
    * The search query string (optional. mastering the search query cmd is a dark art.)
 The script will then:
    * Find the OneDrive for Business site for each user in the text file
    * Create and start a Content Search using the above information
#>
Import-module ExchangeOnlineManagement
Import-Module Microsoft.Online.SharePoint.PowerShell
Connect-SPOService -Credential $creds -Url https://magdev-admin.sharepoint.com -ModernAuth $true -AuthenticationUrl https://login.microsoftonline.com/organizations
Connect-IPPSSession
# Get other required information
$script:inputfile = read-host "Enter the file name of the text file that contains the email addresses for the users you want to search"
$searchName = Read-Host "Enter the name for the new search"
$tempDir = "C:\Temp"
New-Item $tempDir\ODUrls.txt
ForEach ($emailAddress in Get-Content $script:inputfile)
{
    $OneDriveURL = Get-SPOSite -IncludePersonalSite $true -Limit all -Filter "Owner -like $emailAddress" | Select-Object -ExpandProperty Url 
    if ($null -ne $OneDriveURL){ 
        Add-content $tempDir\ODUrls.txt $OneDriveURL
        Write-Host "$emailAddress => $OneDriveURL"
    } else {
        Write-Warning "Could not locate OneDrive for $emailAddress"
    }
}
$emailAddresses = Get-Content $inputfile | Where-Object {$_ -ne ""}  | ForEach-Object{ $_.Trim() }
$urls = Get-Content $tempDir\ODUrls.txt | Where-Object {$_ -ne ""}  | ForEach-Object{ $_.Trim() }
Write-Host "Creating and starting the search"
# Collect OneDrive & Email
$search = New-ComplianceSearch -Name $searchName -ExchangeLocation $emailAddresses -SharePointLocation $urls -ContentMatchQuery $searchQuery
# Finally, start the search and then display the status
if($search)
{
    Start-ComplianceSearch $search.Name
    Get-ComplianceSearch $search.Name
} 
Remove-Item $tempDir\ODUrls.txt

MS One Drive: MSOneDriveSearch.ps1

Note: you will get 2 authentication prompts as you are logging on to Security & Compliance Center as well as the Sharepoint Admin panel.

<# MS OneDrive Security & Compliance Search 
version 2.0
https://github.com/dwmetz/Axiom-PowerShell
Author: @dwmetz
Function:

Function: This script will generate a Security and Compliance Search to capture OneDrive for a list of custodians.

This PowerShell script will prompt you for the following information:
    * Your user credentials                                          
    * The pathname for the text file that contains a list of user email addresses
    * The name of the Content Search that will be created
    * The search query string (optional. mastering the search query cmd is a dark art.)
 The script will then:
    * Find the OneDrive for Business site for each user in the text file
    * Create and start a Content Search using the above information
#>
Import-module ExchangeOnlineManagement
Import-Module Microsoft.Online.SharePoint.PowerShell
Connect-SPOService -Credential $creds -Url https://magdev-admin.sharepoint.com -ModernAuth $true -AuthenticationUrl https://login.microsoftonline.com/organizations
Connect-IPPSSession
# Get other required information
$script:inputfile = read-host "Enter the file name of the text file that contains the email addresses for the users you want to search"
$searchName = Read-Host "Enter the name for the new search"
$tempDir = "C:\Temp"
New-Item $tempDir\ODUrls.txt
ForEach ($emailAddress in Get-Content $script:inputfile)
{
    $OneDriveURL = Get-SPOSite -IncludePersonalSite $true -Limit all -Filter "Owner -like $emailAddress" | Select-Object -ExpandProperty Url 
    if ($null -ne $OneDriveURL){ 
        Add-content $tempDir\ODUrls.txt $OneDriveURL
        Write-Host "$emailAddress => $OneDriveURL"
    } else {
        Write-Warning "Could not locate OneDrive for $emailAddress"
    }
}
$urls = Get-Content $tempDir\ODUrls.txt | Where-Object {$_ -ne ""}  | ForEach-Object{ $_.Trim() }
# Collect OneDrive 
$search = New-ComplianceSearch -Name $searchName -SharePointLocation $urls -ContentMatchQuery $searchQuery
# Finally, start the search and then display the status
if($search)
{
    Start-ComplianceSearch $search.Name
    Get-ComplianceSearch $search.Name
} 
Remove-Item $tempDir\ODUrls.txt

*Microsoft Teams

There are 2 scripts here for Microsoft Teams. Note – by default a Mailbox .pst file that contains Teams data, will not show that Teams data when the .pst is viewed with Outlook. Magnet AXIOM easily parses the Teams content, whether integrated as part of a mailbox collection, or from collections where just MS Teams data is captured.

MS Teams – single custodian: MSTeamsSearch.ps1

<# MS Teams Security & Compliance Search 
version 2.0
https://github.com/dwmetz/Axiom-PowerShell
Author: @dwmetz
Function:
    Collect an O365 mailbox search for MS Teams communications.
    Note this script requires previous installation of the ExchangeOnlineManagement PowerShell module
    See https://docs.microsoft.com/en-us/powershell/exchange/connect-to-scc-powershell?view=exchange-ps for more information.
    
Updates:
    25.October.2022 - updated ExchangeOnlineManagement connection, Security & Compliance Center (IPPSSession)
    
#>
    Import-module ExchangeOnlineManagement
    Connect-ExchangeOnline
    [string]$aname = Read-Host -Prompt 'Enter your account name'
    Connect-IPPSSession -UserPrincipalName $aname
    [string]$name = Read-Host -Prompt 'Enter a name for the search'
    [string]$email = Read-Host -Prompt 'Enter the users email address'
    new-compliancesearch -name $name -ExchangeLocation $email -ContentMatchQuery 'kind=microsoftteams','ItemClass=IPM.Note.Microsoft.Conversation','ItemClass=IPM.Note.Microsoft.Missed','ItemClass=IPM.Note.Microsoft.Conversation.Voice','ItemClass=IPM.Note.Microsoft.Missed.Voice','ItemClass=IPM.SkypeTeams.Message'
    Start-ComplianceSearch $name
    Get-ComplianceSearch $name
    Write-Host "Search initiated."-ForegroundColor Cyan
    Write-Host "Proceed to https://protection.office.com/ to download the results."-ForegroundColor Cyan

MS Teams – group of custodians: MSTeamsGroupSearch.ps1

<# MS Teams (Group) Security & Compliance Search 
version 1.0
https://github.com/dwmetz/Axiom-PowerShell
Author: @dwmetz
Function:
    Collect MS Teams for group of custodians in O365.
    Note this script requires previous installation of the ExchangeOnlineManagement PowerShell module
    See https://docs.microsoft.com/en-us/powershell/exchange/connect-to-scc-powershell?view=exchange-ps for more information.
      
This PowerShell script will prompt you for the following information:
    * Your user credentials                                          
    * The pathname for the text file that contains a list of user email addresses

The script will then:
    * Create and start a Content Search using the above information

Updates:
    17.November.2022 - ExchangeOnlineManagement connection, Security & Compliance Center (IPPSSession)

#>
# New Auth
Import-module ExchangeOnlineManagement
Connect-IPPSSession
# Get other required information
$inputfile = read-host "Enter the file name of the text file that contains the email addresses for the users you want to search"
$searchName = Read-Host "Enter the name for the new search"
$emailAddresses = Get-Content $inputfile | Where-Object {$_ -ne ""}  | ForEach-Object{ $_.Trim() }
Write-Host "Creating and starting the search"
$search = New-ComplianceSearch -Name $searchName -ExchangeLocation $emailAddresses -ContentMatchQuery 'kind=microsoftteams','ItemClass=IPM.Note.Microsoft.Conversation','ItemClass=IPM.Note.Microsoft.Missed','ItemClass=IPM.Note.Microsoft.Conversation.Voice','ItemClass=IPM.Note.Microsoft.Missed.Voice','ItemClass=IPM.SkypeTeams.Message'
# Finally, start the search and then display the status
if($search)
{
    Start-ComplianceSearch $search.Name
    Get-ComplianceSearch $search.Name
} 
Write-Host "Search initiated."-ForegroundColor Cyan
Write-Host "Proceed to https://protection.office.com/ to download the results."-ForegroundColor Cyan

All of the scripts above can be downloaded from my Axiom-PowerShell GitHub repo. You can grab all the scripts at once by going to the latest releases file.