Error pwd.getpwuid with google cloud storage
<p>I <code>pip install --upgrade google-cloud-storage -t libs</code> to my app engine app.</p>

<p>In the appengine_config.py, I added: </p>

<pre><code>vendor.add('libs')
vendor.add(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'libs'))
</code></pre>

<p>It works on app engine online, but not on app engine sandbox locally.</p>

<pre><code>ERROR    2016-10-03 00:22:01,311 wsgi.py:263] 
Traceback (most recent call last):
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
    handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
    handler, path, err = LoadObject(self._handler)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
    obj = __import__(path[0])
  File "/Users/charlesng/Documents/Codes/python/web/myapp/src/main.py", line 19, in &lt;module&gt;
    from handlers import page_handlers, user_handlers, repo_handlers, doc_handlers
  File "/Users/charlesng/Documents/Codes/python/web/myapp/src/handlers/repo_handlers.py", line 28, in &lt;module&gt;
    from google.cloud import storage
  File "/Users/charlesng/Documents/Codes/python/web/myapp/src/libs/google/cloud/storage/__init__.py", line 42, in &lt;module&gt;
    from google.cloud.storage.batch import Batch
  File "/Users/charlesng/Documents/Codes/python/web/myapp/src/libs/google/cloud/storage/batch.py", line 29, in &lt;module&gt;
    from google.cloud.exceptions import make_exception
  File "/Users/charlesng/Documents/Codes/python/web/myapp/src/libs/google/cloud/exceptions.py", line 24, in &lt;module&gt;
    from google.cloud._helpers import _to_bytes
  File "/Users/charlesng/Documents/Codes/python/web/myapp/src/libs/google/cloud/_helpers.py", line 62, in &lt;module&gt;
    _USER_ROOT = os.path.expanduser('~')
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py", line 262, in expanduser
    userhome = pwd.getpwuid(os.getuid()).pw_dir
KeyError: 'getpwuid(): uid not found: 429123195'
</code></pre>

<p>Folder structures:</p>

<pre><code>myapp/
   /src/main.py
   /src/libs
   /env/(virtualenv files)
   /env/lib
</code></pre>

<p>normally if you pip a library, the files are in lib/ but for app engine third party library, we have to pip -t libs so they are in libs instead of lib. </p>

<p>when I use python2 or python3, from google.cloud import storage, they are good but not running the appengine sandbox because it's calling src/libs/google/cloud instead of env/lib/google/cloud.</p>

<p>How should I approach to solve this issue? Any advice or direction would be thankful.</p>

<p>Not sure if this entirely answers your question, however, I am assuming that you are using <a href="https://cloud.google.com/appengine/docs/python/download" rel="nofollow">Google App Engine SDK</a> to launch the app locally on your machine. And this, uses a patched <code>os</code> module for sandboxing:</p>

<pre><code>/tmp/google_appengine$ cat google/appengine/tools/devappserver2/python/sandbox.py
...
...
...

  def apply_policy(self, module_dict):
    """Apply this policy to the provided module dict.

    In order, one of the following will apply:
    - Symbols in overrides are set to the override value.
    - Symbols in deletes are removed.
    - Whitelisted symbols and symbols with a constant type are unchanged.
    - If a default stub is set, all other symbols are replaced by it.
    - If default_pass_through is True, all other symbols are unchanged.
    - If default_pass_through is False, all other symbols are removed.

    Args:
      module_dict: The module dict to be filtered.
    """
    for symbol in module_dict.keys():
      if symbol in self.overrides:
        module_dict[symbol] = self.overrides[symbol]
      elif symbol in self.deletes:
        del module_dict[symbol]
      elif not (symbol in self.whitelist or
                isinstance(module_dict[symbol], self.constant_types) or
                (symbol.startswith('__') and symbol.endswith('__'))):
        if self.default_stub:
          module_dict[symbol] = self.default_stub
        elif not self.default_pass_through:
          del module_dict[symbol]

_MODULE_OVERRIDE_POLICIES = {
    'os': ModuleOverridePolicy(
        default_stub=stubs.os_error_not_implemented,
        whitelist=['altsep', 'curdir', 'defpath', 'devnull', 'environ', 'error',
                   'fstat', 'getcwd', 'getcwdu', 'getenv', '_get_exports_list',
                   'name', 'open', 'pardir', 'path', 'pathsep', 'sep',
                   'stat_float_times', 'stat_result', 'strerror', 'sys',
                   'walk'],
        overrides={
            'access': stubs.fake_access,
            'listdir': stubs.RestrictedPathFunction(os.listdir),
            # Alias lstat() to stat() to match the behavior in production.
            'lstat': stubs.RestrictedPathFunction(os.stat),
            'open': stubs.fake_open,
            'stat': stubs.RestrictedPathFunction(os.stat),
            'uname': stubs.fake_uname,
            'getpid': stubs.return_minus_one,
            'getppid': stubs.return_minus_one,
            'getpgrp': stubs.return_minus_one,
            'getgid': stubs.return_minus_one,
            'getegid': stubs.return_minus_one,
            'geteuid': stubs.return_minus_one,
            'getuid': stubs.return_minus_one,
            'urandom': stubs.fake_urandom,
            'system': stubs.return_minus_one,
            },
        deletes=['execv', 'execve']),
    'signal': ModuleOverridePolicy(overrides={'__doc__': None}),
    'locale': ModuleOverridePolicy(
        overrides={'setlocale': stubs.fake_set_locale},
        default_pass_through=True),
    'distutils.util': ModuleOverridePolicy(
        overrides={'get_platform': stubs.fake_get_platform},
        default_pass_through=True),
    # TODO: Stub out imp.find_module and friends.
    }
</code></pre>

<p>And as you can see, os.getuid() will always return -1:</p>

<pre><code>/tmp/google_appengine$ grep -A1 return_minus_one google/appengine/tools/devappserver2/python/stubs.py
def return_minus_one(*unused_args, **unused_kwargs):
  return -1
</code></pre>

<p>And <code>-1</code> gets converted to <code>429123195</code> because, in python source code (Modules/pwdmodule.c)...</p>

<pre><code>static PyObject *
pwd_getpwuid(PyObject *self, PyObject *args)
{
    uid_t uid;
    struct passwd *p;
    if (!PyArg_ParseTuple(args, "O&amp;:getpwuid", _Py_Uid_Converter, &amp;uid)) {
        if (PyErr_ExceptionMatches(PyExc_OverflowError))
            PyErr_Format(PyExc_KeyError,
                         "getpwuid(): uid not found");
        return NULL;
    }
    if ((p = getpwuid(uid)) == NULL) {
        if (uid &lt; 0)
            PyErr_Format(PyExc_KeyError,
                         "getpwuid(): uid not found: %ld", (long)uid);
        else
            PyErr_Format(PyExc_KeyError,
                         "getpwuid(): uid not found: %lu", (unsigned long)uid);
        return NULL;
    }
    return mkpwent(p);
}
</code></pre>

<p>... uid_t is type-casted to long</p>

<p>As of today (03-10-2016), the <a href="https://cloud.google.com/appengine/kb/" rel="nofollow">google app engine knowledge base</a> article says (under the Python section):</p>

<blockquote>
  <p>The system does not allow you to invoke subprocesses, as a result some
  os module methods are disabled</p>
</blockquote>

python google-app-engine google-cloud-storage