[Python] multiprocessingを試す (3)

前回からの続きです。

[Python] multiprocessingを試す (2)

以下の記事の続きです。 Python の multiprocessing で lock を使ってみます。 multiprocessing の際にリソースを共有した場合、lock をしておかないと変なことになるという話です。 lock lock...

yottagin.com

2019-07-12 15:32

multiprocessing の log 記録の方法です。

ログ記録

multiprocessing.get_logger() と multiprocessing.log_to_stderr() を使うことで、プロセスごとの log を簡単に処理できるようになります。

前回のコードを少し変更して、logを出力してみます。

import logging
import multiprocessing
import time

def add_100_lock(total, lock):
    multiprocessing.get_logger().info('add process started')
    for i in range(100):
        time.sleep(0.01)
        lock.acquire()
        total.value += 1
        lock.release()
    multiprocessing.get_logger().info('add process finished')

def sub_100_lock(total, lock):
    multiprocessing.get_logger().info('sub process started')
    for i in range(100):
        time.sleep(0.01)
        lock.acquire()
        total.value -= 1
        lock.release()
    multiprocessing.get_logger().info('sub process finished')

if __name__ == '__main__':
    multiprocessing.log_to_stderr()
    logger = multiprocessing.get_logger()
    logger.setLevel(logging.INFO)
    
    start = time.time()

    total = multiprocessing.Value('i', 1000)
    lock = multiprocessing.Lock()
    
    add_process = multiprocessing.Process(target=add_100_lock, args=(total, lock))
    sub_process = multiprocessing.Process(target=sub_100_lock, args=(total, lock)) 
    
    add_process.start()
    sub_process.start()

    multiprocessing.get_logger().info('mainprocess loop started')
    for i in range(100):
        time.sleep(0.01)
    multiprocessing.get_logger().info('mainprocess loop finished')

    add_process.join()
    sub_process.join()
    multiprocessing.get_logger().info('process joined')

    print(total.value)

    elapsed_time = time.time() - start
    print (f"elapsed_time:{elapsed_time}[sec]")


# 出力
# [INFO/MainProcess] allocating a new mmap of length 4096
# [INFO/MainProcess] mainprocess loop started
# [INFO/Process-1] child process calling self.run()
# [INFO/Process-1] add process started
# [INFO/Process-2] child process calling self.run()
# [INFO/Process-2] sub process started
# [INFO/MainProcess] mainprocess loop finished
# [INFO/Process-1] add process finished
# [INFO/Process-1] process shutting down
# [INFO/Process-1] process exiting with exitcode 0
# [INFO/Process-2] sub process finished
# [INFO/Process-2] process shutting down
# [INFO/Process-2] process exiting with exitcode 0
# [INFO/MainProcess] process joined
# 1000
# elapsed_time:1.252216100692749[sec]
# [INFO/MainProcess] process shutting down

MainProcess、Process-1、Process-2と、自動的にプロセスごとにログを分けてくれています。