I’m trying to create remote library for library that needs to interact with external service and it didn’t work well when used together with pabot (if not errors then I often asked for the same resource in separate processes). I don’t have problems with implementing it but I’m looking for something similiar to async keyword. Here is simplified example:
import time import random class RemoteLibrary: def __init__(self): self.items = dict() def get_item(self, item_no): # quick if item_no in self.items: return self.items[item_no] return generate_item(item_no): def generate_item(self, item_no): time.sleep(10) self.items[item_no] = random.randint(0, 100) return self.items[item_no]
Most of the time item with given item_no is created so I should be able to return it quickly. But in case it doesn’t exist I need to create it and it takes some time - in that situation all my pabot process need to wait even if they need different item that already exist.
I rather need keyword that would work asynchronously - only processes that require item that doesn’t yet exist should wait for
generate_item method, rest should get quick response.
I could create something on my own but perhaps someone have done it before?