Python subprocess.Popen and asynchronous output

Question

I have simple Python script to execute test suite both under Windows and Linux. Every test writes its output to separate file. I use subprocess.Popen class to execute shell command in a cycle.

Every shell command starts like that:

def system_execute(self, command, path, out_file):
    params_list = command.split(' ') 
    file_path = os.path.join(path, out_file)
    f = open(file_path, "w")
    subprocess.Popen(params_list, stdout=f)
    f.close()

It works fine, but script finishes its work before all output files have been written. Actually, I getting hundreds of zero-size files, and it takes some time to finish writing output and close handles. Could anyone explain the reason why it works so strange, and is there a synchronous way to do the same job?

Thanks

glglgl · Accepted Answer · 2011-09-28 10:50:23Z

16

Before f.close(), you have to wait() for our subprocess.

def system_execute(self, command, path, out_file):
    params_list = command.split(' ') 
    file_path = os.path.join(path, out_file)
    f = open(file_path, "w")
    sp = subprocess.Popen(params_list, stdout=f)
    sp.wait()
    f.close()

or just

def system_execute(self, command, path, out_file):
    params_list = command.split(' ') 
    file_path = os.path.join(path, out_file)
    f = open(file_path, "w")
    subprocess.call(params_list, stdout=f)
    f.close()

(or, for easier file handling,

[...]
    with open(file_path, "w") as f:
        subprocess.call(params_list, stdout=f)

answered Sep 28, 2011 at 10:50

glglgl

91.7k13 gold badges158 silver badges230 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python subprocess.Popen and asynchronous output

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related