python Python Subprocess.Popen Examples

[Linux Hint]

Python subprocess is one of the useful modules that is used to run different shell commands, processes, and execute another script or executable files using the Python script. It can be used also for redirecting the data from one process to another process and handling the errors that are generated by the child processes. The subprocess module has many classes that are used for various purposes. The “subprocess.Popen” class is one of the classes of this module that is used to interact with the external processes and perform different types of tasks among the processes. Multiple uses of the “subprocess.Popen” class in the Python script are shown in this tutorial.

Topic of Contents:

1. Run a Simple Linux Command
2. Run a Command with Input Data
3. Run a Python Script
4. Handle the Subprocess Error
5. Return the Code of Subprocess.Popen
6. Redirect the Output of the Subprocess to File
7. Multiprocessing Using Subprocess.Popen
8. Handle the Input and Output Streams
9. Control the Timeout and Delay
10. Read a Streaming Input

Run a Simple Linux Command

The “date” command is used to display the current system date and time. Create a Python file with the following script that creates a subprocess that executes the “date” command and print the output of this command:

#Import the module

import subprocess

#Define a command for the subprocess and

#open process by using Popen() function

output = subprocess.Popen(["date"], stdout=subprocess.PIPE)

#Retrieve the output and error by communicating with the process

stdout, stderr = output.communicate()

#Print the output

print(stdout.decode())

Output:

The following output appears after executing the previous script:

Go to top

Run a Command with Input Data

The “wc” command with the “-c” option is used to count the total number of characters of the string value that is provided with this command. Create a Python file with the following script that creates a subprocess with the Popen() function to run the “wc –c” commands. The string value is taken from the terminal after executing the script and the total characters of the input string are printed in the output.

#Import the module

import subprocess

#Define a command for the subprocess and

#Open process by using Popen() function

output = subprocess.Popen(["wc", "-c"], stdout=subprocess.PIPE)

#Retrieve the output and error by communicating with the process

stdout, stderr = output.communicate()

#Print the output

print(stdout.decode())

Output:

The following output appears for the “Python Subprocess Examples” input value:

Go to top

Run a Python Script

Create a Python file named “sum.py” with the following script that calculates the sum of two numbers and these numbers are provided as the command-line arguments:

sum.py

#Import necessary module

import sys

#Count total arguments

n = len(sys.argv)

#Add the first two argument values

sum = int(sys.argv[1]) + int(sys.argv[2])

#Print the addition result

print("The sum of " + sys.argv[1] + " and " + sys.argv[2] + " is", sum)

Create a Python file with the following script that will run a Python file named sum.py with two arguments by creating a subprocess.

#Import the module

import subprocess

#Run python script in the subprocess and

#open the process by using Popen() function

output = subprocess.Popen(["python3", "sum.py", "25", "55"], stdout=subprocess.PIPE)#Retrieve the output and error by communicating with the process

stdout, stderr = output.communicate()

#Print the output

print(stdout.decode())

Output:

The following output appears after executing the previous script:

Go to top

`

Handle the Subprocess Error

Create a Python file with the following script that handles the errors of the subprocess using the “try-except” block. A command is taken from the user and it is executed by the subprocess. The error message is displayed if any invalid command is taken from the user.

#Import the modules

import subprocess

import sys

#Take command from the user

command = input("Enter a valid command: ")

try:

#Open process by using Popen() function

output = subprocess.Popen([command], stdout=subprocess.PIPE)

#Retrieve the output and error by communicating with the process

stdout, stderr = output.communicate()

#Print the output

print (stdout.decode())

except:

print ("Error:", sys.exc_info())

Output:

The following output appears if the “pwd” command is taken as input that is a valid command:

The following output appears if the “usr” command is taken as input that is a valid command:

Go to top

Return the Code of Subprocess.Popen

Create a Python file with the following script that executes an “ls” command through the subprocess to get the list of all Python files from the current location. The script waits to complete the subprocess and prints the return code.

#Import the modules

import subprocess

import sys

#Set the command

command = ['ls', '-l', '*.py']

try:

#Open process by using Popen() function

output = subprocess.Popen(command, stdout=subprocess.PIPE,

stderr=subprocess.PIPE, text=True)

#Wait to complete the process

retCode = output.wait()

#Print the return code

print("Return Code:", retCode)

except:

#Print error message for the wrong

print ("Error:", sys.exc_info())

Output:

A similar output appears after executing the previous script:

Go to top

Redirect the Output of the Subprocess to File

Create a Python file with the following script that writes the output of the subprocess in a text file. The command that is executed by the subprocess is taken from the user.

#Import module

import subprocess

#Define the filename

filename = "outfile.txt"

#Take a ping command

cmd = input("Enter a ping command: ")

#Split the taken input based on the space

args = cmd.split()

#Write the command output in the file

with open(filename, 'w') as outdata:

process = subprocess.Popen(args,stdout=outdata)

#Wait for completing the process

return_code = process.wait()

Output:

According to the following output, the “ping -c 3 www.google.com” command is taken from the user and the “cat” command is used to display the content of the file that is written by the subprocess:

Go to top

Multiprocessing Using Subprocess.Popen

Create a Python file with the following script where the use of multiprocessing is shown using subprocess. Here, a function named display_msg() is called multiple times using multiprocessing.

#Import necessary modules

import multiprocessing

import subprocess

#Define the function that will be called by multiprocessing

def display_msg(n):

#Define the command with the format() function

cmd = "echo 'Python Programming'".format(n)

#Open process by using Popen() function

process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)

#Retrieve the output and error by communicating with the process

stdout, error = process.communicate()

#Print the output

print (stdout.decode())

#Call the function 5 times by creating the multiprocessing.Pool

with multiprocessing.Pool(multiprocessing.cpu_count()) as mp:

#Map the function

mp.map(display_msg, range(1, 5))

Output:

The following output appears after executing the previous script:

Go to top

Handle the Input and Output Streams

Create a text file named “test.txt” with the following content before creating the Python script of this example.

test.txt

PERL

python

bash

php

Create a Python file with the following script that uses one subprocess to read the content of the “test.txt” file and another subprocess to search for a particular word in that text file. Here, the word “python” is searched in the “test.txt file” that contains this word.

#Import modules

import subprocess

#Define a process for the input stream

in_process = subprocess.Popen(["cat", "test.txt"], stdout=subprocess.PIPE, text=Tru>#Define a process for the output stream

out_process = subprocess.Popen(

["grep", "python"], stdin=in_process.stdout,

stdout=subprocess.PIPE, text=True)

#Store the output of the input and output processes

output, _ = out_process.communicate()

#Print the output

print ("Output:", output)

Output:

The following output appears after executing the script:

Go to top

Control the Subprocess Using a Timer

Create a Python file with the following script that uses a timer to execute a command using a subprocess. Here, the “try-except” block is used to start the timer and the “finally” block is used to cancel the timer.

#Import the subprocess module

import subprocess

#Import the Timer module

from threading import Timer

#Define the command

cmd = ['ping', 'www.example.com']

#Open the process

p = subprocess.Popen(

cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

#Define the timer

timer = Timer(2, lambda process: process.kill(), [p])

try:

#Start the timer

timer.start()

#Read output

stdout, _ = p.communicate()

#Print output

print (stdout.decode())

except:

#Print error message for the wrong

print ("Error:", sys.exc_info())

finally:

#Stop the timer

timer.cancel()

Output:

The following output appears after executing the script:

Go to top

Read the Streaming Input

Create a Python file that reads the content of the subprocess output using a “while” loop and store the content into a variable. The content of this variable is printed later. Here, the “curl” command is used in the subprocess for the www.google.com URL.

#Import module

import subprocess

#Define command

cmd = ["curl", "www.example.com"]

p = subprocess.Popen(cmd, stdout=subprocess.PIPE,

stderr=subprocess.PIPE, text=True>

#Initialize the output variable

output = ""

while True:

#Read the process output line by line

ln = p.stdout.readline()

#Terminate from the loop when the subprocess finishes

if not ln:

break

output = output + ln

#Print the line

print (output)

#Get the return code after finishing the process

return_code = p.wait()

#Print the return code

print ("Return code: ", return_code)

Output:

The last part of the three outputs is shown in the following image. The return code after completing the subprocess is 0:

Go to top

Conclusion

Different uses of the Python subprocess.Popen() are shown in this tutorial using multiple Python scripts that will help the Python users to know the basic uses of this function.

View the full article