python external programs

Python: Running External Programs

by

Python is more than just a language for writing code—it’s a powerful tool for automation, scripting, and system integration. One of its most useful capabilities is the ability to run external programs directly from your Python scripts.

This becomes essential when you need to:

  • Execute system commands like ls, dir, or mkdir
  • Automate interactions with tools like git, ffmpeg, or curl
  • Launch other Python scripts or programs as part of a larger workflow

Whether you’re building developer utilities, scripting deployment processes, or simply chaining tasks together, running external programs with Python opens the door to more dynamic and flexible solutions.

In this guide, we’ll focus purely on the how-to—clear techniques, hands-on examples, and well-structured code you can use right away. By the end, you’ll be able to confidently use Python to launch and manage external programs as part of your everyday scripts and tools.

Using os.system()

The os.system() function provides a straightforward way to execute an external command from within a Python script. When you call os.system(), Python passes the command string to the operating system’s shell, which runs it as if you had typed it directly into a terminal or command prompt.

For example, a simple use case is to print a message using the shell’s echo command:

import os

os.system("echo Hello from the Shell!")

This executes the echo command, and the message is displayed directly in the console.

Another common use of os.system() is to check the version of Python installed on the system. This can be done by running the command python --version or python3 --version depending on your environment:

import os

os.system("python --version")

This command will output the installed Python version directly to the terminal.

While os.system() is easy to use and useful for quick tasks, it has some limitations. It returns only the exit status of the command, not the output generated by it. Therefore, if you need to capture the output of the external program or interact with it more flexibly, os.system() is not the ideal choice. For such cases, the subprocess module offers much greater control and will be covered in the following sections.

Using subprocess.run()

Starting with Python 3.5, the recommended way to run external programs is by using the subprocess.run() function. It provides a safer and more powerful interface than os.system(), and gives you more control over how commands are executed and how their output is handled.

To run a basic shell command, you can pass a list of arguments to subprocess.run(), where each part of the command is a separate string. For example, to run the echo command and print a message, you can use the following code:

import subprocess

subprocess.run(["echo", "Hello from subprocess!"], shell=True)

This launches the command as if it were typed into the shell, and prints the output to the console.

You can also use subprocess.run() to execute other Python scripts. This is particularly useful when building tools that need to orchestrate multiple scripts or automate tasks. The example below shows how to run a Python file named myscript.py:

import subprocess

subprocess.run(["python", "myscript.py"])

This will execute the script using the Python interpreter available on your system. Note that the command should match your environment—on some systems, it may be necessary to use python3 instead of python.

Capturing Output with subprocess

One of the key advantages of the subprocess module over os.system() is its ability to capture the output of a command. This allows you to store the result of an external program in a variable, process it within your Python script, or use it in conditional logic.

To capture output, you can use the capture_output=True argument with subprocess.run(), along with text=True to receive the output as a string instead of raw bytes.

For example, the following code captures the output of the python --version command and prints it:

import subprocess

result = subprocess.run(["python", "--version"], capture_output=True, text=True)

print("Running Python Version:", result.stdout)

In this case, result.stdout contains the standard output of the command. If there were any errors, they would be available in result.stderr. This approach is especially useful when working with tools or scripts that return structured data or logs you need to inspect or manipulate programmatically.

Capturing output gives you a much greater level of control and opens the door to more dynamic and responsive Python applications. In the next section, we’ll take this further by using subprocess.Popen() for advanced interaction with external processes.

Using subprocess.Popen() for More Control

While subprocess.run() is suitable for most use cases, there are situations where more control over the execution is needed—especially when you want to interact with a process while it’s running, stream output, or handle input and output separately. For these cases, subprocess.Popen() offers a more flexible and powerful interface.

The Popen class allows you to start a process and manage its input, output, and error streams directly. This is useful when you need to read output incrementally, send input dynamically, or keep the process running in the background.

The following example demonstrates how to run a small inline Python command using Popen, capture its output, and print the result:

from subprocess import Popen, PIPE

process = Popen(["python", "-c", "print('Python Rocks!')"], stdout=PIPE)
output = process.communicate()[0]

print(output.decode())

In this example, Popen runs a one-line Python command using the -c flag. The stdout=PIPE argument tells Python to capture the output stream. The communicate() method waits for the process to complete and returns a tuple containing the output and error streams. The result is decoded from bytes to a string before being printed.

Popen provides a fine-grained level of control over process execution, making it ideal for more complex automation tasks or when integrating long-running or interactive commands into your Python programs.

Sending Input to Programs

In addition to capturing output, the subprocess module also allows you to send input to a running program. This is particularly useful when working with interactive scripts or command-line tools that expect user input during execution.

Using subprocess.Popen() with the stdin=PIPE argument, you can send data to the process through its standard input stream. This allows your Python script to programmatically provide the input that the external program would normally receive from a user.

The following example demonstrates how to run a short Python command that reads input and prints it. The input is sent from your script using communicate():

from subprocess import Popen, PIPE

process = Popen(["python", "-c", "print(input())"], stdin=PIPE, stdout=PIPE)
out, _ = process.communicate(b'Hello world!')

print("Output:", out.decode())

In this code, the -c flag runs a simple inline Python script that prints the result of input(). The input text "Hello world!" is sent to the process via communicate(), and the resulting output is captured and decoded.

This approach is effective for automating interactions with programs that require input, allowing your Python script to act as both the sender and receiver in a simulated user session.

Sending Multiple Inputs to a Script

Sometimes, a script requires more than one input—just like a user might enter several responses in a terminal. With subprocess.Popen, you can send multiple lines of input by formatting them as a single byte string, using newline characters (\n) to separate each input.

Let’s start with a simple script that expects two inputs:

# myscript.py

name = input("What’s your name? ")
language = input("What’s your favorite programming language? ")

print(f"Hello, {name}!")
print(f"{language} is a great choice.")

To send inputs to this script programmatically, use the following Python runner:

from subprocess import Popen, PIPE

input_data = b'Edward\nPython\n'

process = Popen(["python", "myscript.py"], stdin=PIPE, stdout=PIPE, stderr=PIPE)
out, err = process.communicate(input=input_data)

if out:
    print("Output:")
    print(out.decode('mbcs'))

if err:
    print("Errors:")
    print(err.decode('mbcs'))

In this example, the input data simulates a user typing “Edward” and pressing Enter, followed by “Python” and another Enter. The script receives these inputs through stdin and proceeds as if a real user had typed them.

This technique is useful for automating tests, feeding data into command-line tools, or building your own interfaces around interactive scripts.

Running Commands in the Shell

There are times when you may want to run a command exactly as you would type it in a terminal, using features like wildcards, pipes, or command chaining. In such cases, you can enable shell interpretation by setting the shell=True parameter in subprocess.run() or other subprocess functions.

When shell=True is used, the command is passed to the system shell for execution. This enables the use of shell syntax such as |, &&, *, and other features typically available in a command-line environment.

For example, the following command lists all files in the current directory and filters for those ending in .py:

import subprocess

subprocess.run("ls -l | grep .py", shell=True)

This tells the shell to list directory contents with ls -l and then pipe the output to grep to match .py files.

While using the shell can be convenient, it’s important to construct command strings carefully—especially if any part of the input comes from users or external sources—to avoid security issues such as shell injection.

In general, use shell=True only when shell-specific features are needed. For most structured tasks, passing arguments as a list without the shell is safer and more predictable.

Running Scripts with Arguments

Python’s subprocess module makes it easy to run scripts and pass arguments to them, just as you would from the command line. This is especially useful when automating scripts that accept parameters to modify their behavior or process different input.

To pass arguments, simply include them as additional elements in the list you provide to subprocess.run() or another subprocess function. Each part of the command and its arguments should be a separate string.

For example, the following code runs a Python script named myscript.py and passes it two arguments, arg1 and arg2:

import subprocess

subprocess.run(["python", "myscript.py", "arg1", "arg2"])

Inside myscript.py, you can access these arguments using the sys.argv list, where sys.argv[0] is the script name, and sys.argv[1] and onward are the passed arguments.

This approach gives your automation and scripting workflows greater flexibility, allowing you to reuse the same script with different input values or configurations, all managed seamlessly from your main Python program.

Final Example: A Basic Python Runner

To bring everything together, let’s build a simple Python runner. This utility reads a Python file, executes it as a subprocess, captures its output, and displays both the standard output and any errors that occur. Such a tool is useful for testing scripts, automating tasks, or creating developer utilities. You can even use this as a foundation to develop your own Python IDE.

Here’s the complete code for the runner:

import subprocess
import sys

def run_python_file(filename):

    try:

        result = subprocess.run(
            [sys.executable, filename],
            capture_output=True,
            text=True
        )

        print("📤 Output:")
        print(result.stdout)

        if result.stderr:
            print("⚠️ Errors:")
            print(result.stderr)

    except FileNotFoundError:
        print("❌ File not found!")


# Test it
run_python_file("hello_world.py")

This function uses sys.executable to ensure it runs the file with the same Python interpreter as the script itself. It captures both standard output and error streams using capture_output=True, and prints them after execution. If the specified file doesn’t exist, it catches the FileNotFoundError and prints an appropriate message.

To test this runner, create a simple Python file named hello_world.py in the same directory:

# hello_world.py

print("Hello from inside hello_world.py!")

name = "Edward"
print(f"Nice to meet you, {name}!")

When you run the main runner script, it will execute hello_world.py, capture the output, and display it neatly in the console. This example demonstrates how Python can manage and interact with other Python programs, wrapping up our guide with a practical and useful tool.

Conclusion

Running external programs with Python is both simple and powerful. Whether you’re executing basic shell commands, launching scripts with arguments, or capturing output and interacting with processes, Python provides all the tools you need. The subprocess module stands out as the most robust and flexible option, giving you fine control over how external programs are run and integrated into your projects.

With what you’ve learned, you’re well-equipped to build tools, automate workflows, and develop Python scripts that can communicate with the world beyond their own runtime. From quick utilities to complex orchestration tasks, Python makes it easy to get things done.

Now it’s your turn—experiment, explore, and bring your ideas to life.