Python - How can I pass multiple arguments using Pool map
<p>This is a sniplet of my code:</p>

<pre><code>data = [currentAccount.login,currentAccount.password,campaign.titlesFile,campaign.licLocFile,campaign.subCity,campaign.bodiesMainFile,campaign.bodiesKeywordsFile,campaign.bodiesIntroFile]
results = multiprocessing.Pool(5).map(partial(self.postAd,data),range(3))
...
def postAd (self,login,password,titlesFile,licLocFile,subCity,bodiesMainFile,bodiesKeywordsFile,bodiesIntroFile):
...
</code></pre>

<p>(Just so you know what's going on: currentAccount and campaign are classes, those are variables within those classes. Using self b/c this is all being run in a class. I'm trying to run self.postAd 3x passing it all the variables I have in data)</p>

<p>When I run that it says " postAd() missing 6 required positional arguments: 'titlesFile', 'licLocFile', 'subCity', 'bodiesMainFile', 'bodiesKeywordsFile', and 'bodiesIntroFile'"</p>

<p>What am I doing wrong? Why does it only accept 2 variables?</p>

<p>If I cant use Pool map, how should I be doing this?</p>

<p>I also tried this with no success:</p>

<pre><code>results = multiprocessing.Pool(5).map(lambda args: self.postAd(currentAccount.login,currentAccount.password,campaign.titlesFile,campaign.licLocFile,campaign.subCity,campaign.bodiesMainFile,campaign.bodiesKeywordsFile,campaign.bodiesIntroFile), range(3))

Error: Can't pickle &lt;function NewPostService.processNewAds.&lt;locals&gt;.&lt;lambda&gt; at 0x0000000002F3CBF8&gt;: attribute lookup &lt;lambda&gt; on functions failed
</code></pre>

<p>Your first attempt is a misuse of <code>partial</code>. <code>data</code> is a single argument: it being a list doesn't automatically unpack its contents. <code>partial</code> simply takes variable arguments and so you should pass those arguments 'normally', either </p>

<pre><code>partial(self.postAd, currentAccount.login,currentAccount.password, ...
</code></pre>

<p>or</p>

<pre><code>partial(self.postAd, *data)
</code></pre>

<p>The reason it says that <code>postAd</code> received two arguments instead of just one (<code>data</code>) is that it also implicitly received the <code>self</code> argument.</p>

<p>Your second attempt is the right idea and very close, but it so happens that <a href="http://bugs.python.org/issue19272" rel="nofollow">in older versions of python pickle (which is essential for multiprocessing) can't handle lambdas</a>. Replace the lambda with a named function defined using <code>def</code>:</p>

<pre><code>def postNow(arg):  # Accepts an argument from map which it ignores
    self.postAd(currentAccount.login, ..., campaign.bodiesIntroFile)
</code></pre>

<p>Side notes:</p>

<p>It's a bit odd that your argument to the lambda is called <code>args</code> when it's just one argument. If your intention is to make it flexible and accept variable arguments, use <code>lambda *args: ...</code> or even <code>lambda *args, **kwargs: ...</code> to accept keyword arguments. The names are not important, they are just convention. The important part is the <code>*</code> and <code>**</code>. Note that <code>partial</code> has a signature like this, as an example.</p>

<p>I have never seen this issue with the second attempt before. I learnt it by Googling the error message. Always Google things.</p>

python multiprocessing