Python

Overview

The Python Action is a new transformation within the Transformation Pipelines framework that empowers SOC analysts to directly apply custom Python code to log streams ingested from integrated data sources.

Unlike predefined transformation actions (e.g., parsing, field extraction or enrichment) this action provides unlimited flexibility: analysts can implement their own logic to extract, convert, or enrich events in real time, tailored to specific customer requirements or complex use cases.

Python code embedded IDE

Key Capabilities

• Custom code execution: analystsf can write and execute arbitrary Python code directly within the platform.

• Real-time transformation: code is applied to each event in the log stream, enabling dynamic parsing and enrichment.

• Integrated development interface: dedicated screen to:

  • Insert Python code.

  • Test the transformation against sample logs.

  • Preview outputs before production deployment.

• Safe deployment workflow: transformations can be tested and validated prior to being enabled in production pipelines.

• Multi-client applicability: transformations can be scoped to a specific customer’s log sources or reused across multiple tenants where applicable.

---

Security Model and Restrictions

To protect the platform and ensure no malicious activities can be performed by users, the Python runtime enforces a set of constraints to ensure a malicious user has not access to the underlying system. Here you have some of the restrictions:

• No network access: You cannot do outbound or inbound connections.

• No external imports: Only a predefined set of safe Python modules can be used. Importing additional modules is not permitted.

• Restricted CPU and memory usage: Execution time and resource consumption are strictly limited.

• Stateless execution: Each run is isolated, and no state is preserved between executions.

---

Typical Use Cases

• Custom parsing: extracting non-standard fields from vendor-specific logs.

• Format conversion: transforming legacy or proprietary log formats into normalized schemas.

• Field manipulation: cleaning or restructuring field values (string normalization, timestamp correction, data type conversions).

• Business-specific logic: applying customer-specific rules or mappings not covered by the generic transformation catalog.

---

Preloaded Python modules

To support common transformation use cases while maintaining a secure sandbox, the following Python modules are preloaded and available for use:

json

The json module provides utilities for parsing, generating, and manipulating JSON-formatted data. This is especially useful when dealing with logs that contain nested structures or when transforming payloads into different formats.

Typical Capabilities:

• Convert strings into Python dictionaries (json.loads)

• Convert Python dictionaries into JSON strings (json.dumps)

• Safely manipulate JSON fields inside events

Example Usage:

data = json.loads(event["message"])
event["username"] = data.get("user")
event["details_json"] = json.dumps(data)

---

re (Regular Expressions)

The re module offers powerful pattern-matching capabilities, which are critical in log processing environments where fields may need to be extracted, normalized, or validated.

Typical Capabilities:

• Extract structured data from unstructured log messages (re.search, re.findall)

• Replace or normalize values (re.sub)

• Validate patterns such as IP addresses, usernames, or error codes

Example Usage:

match = re.search(r"src_ip=(\d+\.\d+\.\d+\.\d+)", event["message"])
if match:
    event["sourceAddress"] = match.group(1)

---

itertools

The itertools module provides high-performance iterator tools that help with efficient looping, grouping, and combinatorial operations. It is useful in scenarios where events need to be aggregated, deduplicated, or processed in sequences.

Typical Capabilities:

• Chain multiple iterables (itertools.chain)

• Group elements by a specified key (itertools.groupby)

• Create combinatorial sequences (itertools.permutations, itertools.product)

• Build memory-efficient loops for data transformation

Example Usage:

combined = list(itertools.chain(event["tags"], ["processed"]))
event["tags"] = combined

base64

The base64 module provides utilities for encoding and decoding data using Base64, a common representation format in cybersecurity workflows for handling credentials, payloads, binary data, and encoded indicators.

Capabilities:

• Encode byte strings into Base64 (base64.b64encode)

• Decode Base64 data into raw bytes (base64.b64decode)

• Handle common encoding formats used in logs, API payloads, or security alerts

• Process artifacts such as hashed fields, encoded command-and-control data, JWT components, etc.

Example Usage — Decoding Base64 log field:

if "payload_b64" in event:
    decoded_bytes = base64.b64decode(event["payload_b64"])
    event["payload"] = decoded_bytes.decode("utf-8", errors="ignore")

Example Usage — Encoding a field:

encoded = base64.b64encode(event["message"].encode("utf-8"))
event["message_b64"] = encoded.decode("utf-8")

---

How to use it

1

Access a Transformation pipeline of an integrated data source.

2

Add a new Code ⇒ Python transformation to the pipeline from the Transformations component.

3

Click on the new added Python step in the pipelines component: this will open the Python IDE editor.

4

Add an event sample in the Event sample window of the IDE.

5

Write custom Python code in the Python code window.

• Code is executed on the events log stream of the selected integration.

• Analysts can design any manipulation: formatting changes, modifying, adding or deleting fields, transformations, hashing or any transformation they require.

• Use the event variable to reference the event content.

• Use the Try code button (or Alt+Enter) to test the code with the provided event sample and inspect the Output window.

• Adjust code until desired results are achieved.

6

Deploy to production: click on the Apply changes button

From this moment you can check the results in the Console.

---

Security and Best Practices

Code scope: transformations are executed within a controlled environment to prevent impact on platform stability.

Actions like trying to accessing files or external URLs are forbidden for security purposes. If you try to use commands for different purposes than processing the log stream, the executions will be stopped, your activity reported and your tenant will be closed, resulting in the execution of the economic penalties established in the license agreement

Performance considerations: Python code runs for each log event; analysts should ensure transformations are efficient and avoid excessive computation.

---

Example

Sample Log Input (raw):

<134>1 2025-09-20T12:32:14Z firewall-01 app: user=jdoe srcip=192.168.1.50 action=ALLOW Duration=5400

Python Transformation Snippet:

# Extract fields from raw log
    parts = event["message"].split(" ")
    for item in parts:
        if "user=" in item:
            event["userName"] = item.split("=")[1]
        elif "srcip=" in item:
            event["sourceAddress"] = item.split("=")[1]
        elif "action=" in item:
            event["deviceAction"] = item.split("=")[1]
        elif "Duration=" in item:
            event["session_duration"] = int(item.split("=")[1])
    return event

Output Event (transformed):

{
  "timestamp": "2025-09-20T12:32:14Z",
  "device": "firewall-01",
  "username": "jdoe",
  "source_ip": "192.168.1.50",
  "action": "ALLOW",
  "session_duration": 5400
}

---

Conclusion

The Python Action in Transformation Pipelines extends the platform’s flexibility by allowing SOC analysts to implement custom transformations on log streams. This empowers multi-tenant SOCs to adapt rapidly to diverse customer environments, ensuring that all relevant security information is extracted, normalized, and enriched for detection, dashboards, and compliance.

By combining built-in actions with Python-based custom logic, analysts gain full control over the transformation layer, maximizing both operational efficiency and analytical depth.