build fails

[glennra]

This is on Ubuntu 18.04.2. Running in a python3 virtualenv. I have followed the setup instructions in the readme.

`$ pydo :build
--- Logging error ---
Traceback (most recent call last):
File "/home/glenn/devel/rpi-ramdisk/init.py", line 14, in
jobs = int(os.environ['PYDOJOBS'], 10)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/os.py", line 669, in getitem
raise KeyError(key) from None
KeyError: 'PYDOJOBS'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.6/logging/init.py", line 994, in emit
msg = self.format(record)
File "/usr/lib/python3.6/logging/init.py", line 840, in format
return fmt.format(record)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 52, in format
return f'{colors.color(command, **l2c(record.levelno))}: {colors.color(record.msg, fg="white")}'
AttributeError: 'bool' object has no attribute 'color'
Call stack:
File "/home/glenn/devel/rpi-ramdisk/venv/bin/pydo", line 10, in
sys.exit(main())
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/main.py", line 84, in main
importlib.import_module(commands.project_root.name)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 665, in _load_unlocked
File "", line 678, in exec_module
File "", line 219, in _call_with_frames_removed
File "/home/glenn/devel/rpi-ramdisk/init.py", line 18, in
log.warning(f'Setting jobs to {jobs}.')
Message: 'Setting jobs to 4.'
Arguments: ()
Don't forget to set up apt-cacher-ng
Don't forget to set up apt-cacher-ng
Don't forget to set up apt-cacher-ng
--- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.6/logging/init.py", line 994, in emit
msg = self.format(record)
File "/usr/lib/python3.6/logging/init.py", line 840, in format
return fmt.format(record)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 52, in format
return f'{colors.color(command, **l2c(record.levelno))}: {colors.color(record.msg, fg="white")}'
AttributeError: 'bool' object has no attribute 'color'
Call stack:
File "/home/glenn/devel/rpi-ramdisk/venv/bin/pydo", line 10, in
sys.exit(main())
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/main.py", line 106, in main
f()
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 80, in _consider_cmd_and_deps
f()
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 60, in _run_cmd_if_necessary
return f()
File "/home/glenn/devel/rpi-ramdisk/kernel/init.py", line 61, in _build
], env=self.env, shell=True)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 18, in _inner
return f(logger, *args, **kwargs)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/utils.py", line 49, in call
logger.info(c)
Message: 'git -C /home/glenn/devel/rpi-ramdisk/kernel/linux clean -dfxq'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.6/logging/init.py", line 994, in emit
msg = self.format(record)
File "/usr/lib/python3.6/logging/init.py", line 840, in format
return fmt.format(record)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 52, in format
return f'{colors.color(command, **l2c(record.levelno))}: {colors.color(record.msg, fg="white")}'
AttributeError: 'bool' object has no attribute 'color'
Call stack:
File "/home/glenn/devel/rpi-ramdisk/venv/bin/pydo", line 10, in
sys.exit(main())
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/main.py", line 106, in main
f()
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 80, in _consider_cmd_and_deps
f()
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 60, in _run_cmd_if_necessary
return f()
File "/home/glenn/devel/rpi-ramdisk/kernel/init.py", line 61, in _build
], env=self.env, shell=True)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 18, in _inner
return f(logger, *args, **kwargs)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/utils.py", line 49, in call
logger.info(c)
Message: 'rm -rf --one-file-system /home/glenn/devel/rpi-ramdisk/kernel/stage'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.6/logging/init.py", line 994, in emit
msg = self.format(record)
File "/usr/lib/python3.6/logging/init.py", line 840, in format
return fmt.format(record)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 52, in format
return f'{colors.color(command, **l2c(record.levelno))}: {colors.color(record.msg, fg="white")}'
AttributeError: 'bool' object has no attribute 'color'
Call stack:
File "/home/glenn/devel/rpi-ramdisk/venv/bin/pydo", line 10, in
sys.exit(main())
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/main.py", line 106, in main
f()
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 80, in _consider_cmd_and_deps
f()
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 60, in _run_cmd_if_necessary
return f()
File "/home/glenn/devel/rpi-ramdisk/kernel/init.py", line 61, in _build
], env=self.env, shell=True)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 18, in _inner
return f(logger, *args, **kwargs)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/utils.py", line 49, in call
logger.info(c)
Message: 'cp /home/glenn/devel/rpi-ramdisk/kernel/kernel.config /home/glenn/devel/rpi-ramdisk/kernel/linux/.config'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.6/logging/init.py", line 994, in emit
msg = self.format(record)
File "/usr/lib/python3.6/logging/init.py", line 840, in format
return fmt.format(record)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 52, in format
return f'{colors.color(command, **l2c(record.levelno))}: {colors.color(record.msg, fg="white")}'
AttributeError: 'bool' object has no attribute 'color'
Call stack:
File "/home/glenn/devel/rpi-ramdisk/venv/bin/pydo", line 10, in
sys.exit(main())
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/main.py", line 106, in main
f()
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 80, in _consider_cmd_and_deps
f()
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 60, in _run_cmd_if_necessary
return f()
File "/home/glenn/devel/rpi-ramdisk/kernel/init.py", line 61, in _build
], env=self.env, shell=True)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 18, in _inner
return f(logger, *args, **kwargs)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/utils.py", line 49, in call
logger.info(c)
Message: 'make -j4 -C /home/glenn/devel/rpi-ramdisk/kernel/linux zImage modules dtbs'
Arguments: ()
make: *** No rule to make target 'zImage'. Stop.
Traceback (most recent call last):
File "/home/glenn/devel/rpi-ramdisk/venv/bin/pydo", line 10, in
sys.exit(main())
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/main.py", line 106, in main
f()
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 80, in _consider_cmd_and_deps
f()
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 60, in _run_cmd_if_necessary
return f()
File "/home/glenn/devel/rpi-ramdisk/kernel/init.py", line 61, in _build
], env=self.env, shell=True)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 18, in _inner
return f(logger, *args, **kwargs)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/utils.py", line 55, in call
stderr=subprocess.DEVNULL if verbosity < 1 else None,
File "/usr/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'make -j4 -C /home/glenn/devel/rpi-ramdisk/kernel/linux zImage modules dtbs' returned non-zero exit status 2.
`

[ali1234]

Should be fixed in ali1234/pydo@dee856e

It will now fall back to not colouring logs instead of crashing when the colors module is not available.

[glennra]

That works now, but there is another issue:
$ pydo :build
::18: Setting jobs to 4.
Don't forget to set up apt-cacher-ng
Don't forget to set up apt-cacher-ng
Don't forget to set up apt-cacher-ng
[7/15] kernel:_build:61: git -C /home/glenn/devel/rpi-ramdisk/kernel/linux clean -dfxq
[7/15] kernel:_build:61: rm -rf --one-file-system /home/glenn/devel/rpi-ramdisk/kernel/stage
[7/15] kernel:_build:61: cp /home/glenn/devel/rpi-ramdisk/kernel/kernel.config /home/glenn/devel/rpi-ramdisk/kernel/linux/.config
[7/15] kernel:_build:61: make -j4 -C /home/glenn/devel/rpi-ramdisk/kernel/linux zImage modules dtbs
make: *** No rule to make target 'zImage'. Stop.
Traceback (most recent call last):
File "/home/glenn/devel/rpi-ramdisk/venv/bin/pydo", line 10, in
sys.exit(main())
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/main.py", line 106, in main
f()
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 80, in _consider_cmd_and_deps
f()
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 60, in _run_cmd_if_necessary
return f()
File "/home/glenn/devel/rpi-ramdisk/kernel/init.py", line 61, in _build
], env=self.env, shell=True)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 18, in _inner
return f(logger, *args, **kwargs)
File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/utils.py", line 55, in call
stderr=subprocess.DEVNULL if verbosity < 1 else None,
File "/usr/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'make -j4 -C /home/glenn/devel/rpi-ramdisk/kernel/linux zImage modules dtbs' returned non-zero exit status 2.

[ali1234]

That sounds like you did not recursively clone the submodules.

[glennra]

You are right, the submodule update failed:

git submodule update --recursive
remote: Enumerating objects: 29, done.
remote: Counting objects: 100% (29/29), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 16 (delta 13), reused 13 (delta 10), pack-reused 0
Unpacking objects: 100% (16/16), done.
From git://github.com/raspberrypi/linux
97b8d85c9..e20d30ad9 rpi-4.19.y -> origin/rpi-4.19.y
error: Server does not allow request for unadvertised object 3c35cb554e1f0333af3622773afa4013131ba994
Fetched in submodule path 'kernel/linux', but it did not contain 3c35cb554e1f0333af3622773afa4013131ba994. Direct fetching of that commit failed.

[glennra]

I did a shallow clone to save time/space, I guess it needs some commit that I didn't get.

[ali1234]

Ugh that again. Yes, github won't let you shallow clone an arbitrary commit. Only if it is the tip of a branch or a tag. So you have to clone the whole repository. Is this something I need to fix? Because I tried to implement shallow cloning by default before but had to back it out due to this reason. Still looking for a solution (one would be to mirror the upstream repos and just add tags to them).

[glennra]

I think you just need to add a note that shallow cloning won't work for the kernel submodule.
An alternative may be to provide a tarball of the necessary kernel version. With a normal DSL connection (~10Mbps) it still takes a while to download the kernel repo.

[ali1234]

It won't work for most of them unfortunately. I think GNOME were considering enabling it on their gitlab. What I could do is keep the master branch always pointing to an upstream tag. I don't really want to get in the business of mirroring tarballs.