Manjaro-tools
Revision as of 19:06, 5 May 2015 by imported>Excalibur1234
Introduction
manjaro-tools consists of 4 different tools: mkset, buildpkg, buildtree, and buildiso.
mkset and buildpkg include scripts and tools for Manjaro Linux package maintainers. Official packages get built in clean chroots to avoid dependency conflicts with our branches.
buildtree is a little tool to sync Arch abs and Manjaro packages git repositories.
buildiso is a small set of Bash scripts that is capable of building fully functional Manjaro Linux based live medias. It is a very generic tool, so it could potentially be used to generate anything from rescue systems, to install disks, to special interest live CD systems, and who knows what else. Simply put, if it involves Manjaro on a shiny coaster, it can do it.
manjaro-tools is a replacement for devtools and manjaroiso.
A detailed user manual is available at github.
Configuration
manjaro-tools can be configured via editing /etc/manjaro-tools/manjaro-tools.conf
Files
These are the new names for renamed scripts.
- mkmanjaroroot --> mkchroot
- manjarobuild --> buildpkg
- pacstrap --> basestrap
- genfstab --> fstabgen
- arch-chroot --> manjaro-chroot
mkset
mkset is used to create build lists. Build lists can be defined in /etc/manjaro-tools/sets/<buildlistname>.set
To create a build list, one can run in the pkgbuilds dir:
mkset -c <name>
This will create a build list containing all directories with PKGBUILDs in them. This generated list can be then edited, like packages can be removed.
For example, in a git clone of https://github.com/manjaro/packages-openrc
$ ls consolekit/ manjaro-tools-git/ openrc-net/ eudev/ manjaro-tools-git-devel/ openrc-systemdcompat/ eudev-systemdcompat/ manjaro-tools-iso-profiles/ openrc-video/ kde-servicemenus-pkg-tools/ manjaro-tools-livecd/ packages/ keyboardctl-openrc/ networkmanager-openrc/ pkgbuilds-review/ kf5-servicemenus-pkg-tools/ openrc-aur/ polkit-consolekit/ lib32-eudev/ openrc-base/ sddm-consolekit/ lib32-eudev-systemdcompat/ openrc-core/ sysvinit/ lxqt-lightdm-greeter/ openrc-desktop/ upower-pm-utils/ manjaro-openrc/ openrc-devel/ README.md manjaro-tools/ openrc-misc/
Running mkset -c openrc does
$ mkset -c openrc ==> Creating [/etc/manjaro-tools/sets/openrc.set] ... -> Adding consolekit -> Adding eudev -> Adding eudev-systemdcompat -> Adding kde-servicemenus-pkg-tools -> Adding kf5-servicemenus-pkg-tools -> Adding lib32-eudev -> Adding lib32-eudev-systemdcompat -> Adding lxqt-lightdm-greeter -> Adding manjaro-openrc -> Adding manjaro-tools -> Adding manjaro-tools-git -> Adding manjaro-tools-git-devel -> Adding manjaro-tools-iso-profiles -> Adding manjaro-tools-livecd -> Adding networkmanager-openrc -> Adding openrc-aur -> Adding openrc-base -> Adding openrc-core -> Adding openrc-desktop -> Adding openrc-devel -> Adding openrc-misc -> Adding openrc-net -> Adding openrc-systemdcompat -> Adding openrc-video -> Adding polkit-consolekit -> Adding sddm-consolekit -> Adding sysvinit -> Adding upower-pm-utils
buildpkg
buildpkg is used to build a particular package or a set. The help looks like the following for i686
$ buildpkg -h Usage: buildpkg [options] [--] [makepkg args] -p <profile> Set profile or pkg [default: default] -a <arch> Set arch [default: i686] -b <branch> Set branch [default: stable] -r <dir> Chroots directory [default: /srv/manjarobuild] -c Recreate chroot -w Clean up -n Install and run namcap check -s Sign packages -q Query settings and pretend build -h This help
To build a single package, go into the directory which is one above the package build directory (which contains the PKGBUILD), and run it as:
buildpkg -p <package-name>
For example, we want to build the upower-pm-utils package from the following PKGBUILD folders
$ ls consolekit manjaro-tools openrc-video eudev networkmanager-openrc packages eudev-systemdcompat openrc-aur polkit-consolekit kde-servicemenus-pkg-tools openrc-base README.md keyboardctl-openrc openrc-core sddm-consolekit kf5-servicemenus-pkg-tools openrc-desktop sysvinit lib32-eudev openrc-devel upower-pm-utils lib32-eudev-systemdcompat openrc-misc manjaro-openrc openrc-net
To test if the options are correct, the -q option can be used.
$ buildpkg -q upower-pm-utils ==> manjaro-tools -> version: 0.9.5.6 -> manjaro_tools_conf: /etc/manjaro-tools/manjaro-tools.conf ==> OPTIONS: -> arch: i686 -> branch: stable -> chroots: /srv/manjarobuild ==> ARGS: -> mkchrootpkg_args: -r /srv/manjarobuild/stable/i686 -> makepkg_args: -> clean_first: false -> wipe_clean: false -> namcap: false -> sign: false ==> PATHS: -> chrootdir: /srv/manjarobuild/stable/i686 -> profiledir: /etc/manjaro-tools/sets -> pkg_dir: /var/cache/manjaro-tools/stable/i686 -> pacman_conf: /usr/share/manjaro-tools/pacman-default.conf -> makepkg_conf: /usr/share/manjaro-tools/makepkg-i686.conf -> mirrors_conf: /usr/share/manjaro-tools/pacman-mirrors-stable.conf ==> BLACKLIST: -> blacklist_trigger: eudev lib32-eudev upower-pm-utils eudev-systemdcompat lib32-eudev-systemdcompat -> blacklist: libsystemd ==> SETS: -> profiles: default -> profile: upower-pm-utils -> is_profile: false ==> Build queue: -> upower-pm-utils
After that to build the package -p can be used.
$ buildpkg -p upower-pm-utils ==> Updating chroot for [testing] (x86_64)... :: Synchronizing package databases... core is up to date extra is up to date community is up to date :: Starting full system upgrade... there is nothing to do ==> Making package: upower-pm-utils 0.9.23-4 (Mon Jan 12 12:38:16 IST 2015) ==> Retrieving sources... ... ... ==> Leaving fakeroot environment. ==> Finished making: upower-pm-utils 0.9.23-4 (Mon Jan 12 07:08:28 UTC 2015)
To clean the chroot before building -c can be used.
$ buildpkg -p upower-pm-utils -c ==> Creating chroot for [testing] (x86_64)... -> Deleting chroot copy 'aaditya'... -> Deleting chroot copy 'root'... ==> Creating install root at /mnt/datalinux2/manjaro-tools/testing/x86_64/root ==> Installing packages to /mnt/datalinux2/manjaro-tools/testing/x86_64/root :: Synchronizing package databases... core 133.7 KiB 30.8K/s 00:04 [######################] 100% extra 1920.2 KiB 392K/s 00:05 [######################] 100% community 2.7 MiB 512K/s 00:05 [######################] 100% :: There are 25 members in group base-devel: :: Repository core 1) autoconf 2) automake 3) binutils 4) bison 5) fakeroot 6) file 7) findutils 8) flex 9) gawk 10) gcc 11) gettext 12) grep 13) groff 14) gzip 15) libtool 16) m4 17) make 18) pacman 19) patch 20) pkg-config 21) sed 22) sudo 23) texinfo 24) util-linux 25) which ... ... ==> Creating clean working copy [aaditya]... ==> Making package: upower-pm-utils 0.9.23-4 (Mon Jan 12 12:37:00 IST 2015) ==> Retrieving sources... ... ... ==> Leaving fakeroot environment. ==> Finished making: upower-pm-utils 0.9.23-4 (Mon Jan 12 07:07:20 UTC 2015)
To build a set the name of the set can be used. The current sets can be queried with the -q option.
buildtree
buildtree is a little tools to sync arch abs and manjaro packages git repos.
The arguments are:
$ buildtree -h
Usage: buildtree [options]
-s Sync manjaro tree
-a Sync arch abs
-c Clean package tree
-q Query settings
-h This help[/code]
To sync Arch and Manjaro trees:
buildtree -as
buildiso
buildiso is used to build a particular ISO or a set of ISOs.
Overview
The help looks like the following for x86_64:
$ buildiso -h
Usage: buildiso [options]
-p <profile> Buildset or profile [default: default]
-a <arch> Arch [default: x86_64]
-b <branch> Branch [default: unstable]
-r <dir> Chroots directory
[default: /mnt/datalinux2/manjaro-tools/buildiso]
-w Disable clean iso cache
-c Disable clean work dir
-x Disable clean xorg cache
-l Disable clean lng cache
-i Build images only
-s Generate iso only
Requires pre built images (-i)
-q Query settings and pretend build
-h This help
Build profiles for official and community editions of Manjaro can be obtained from here, and this is what the current folder looks like:
$ ls cinnamon kde kde-openrc lxqt-openrc netbook openbox-legacy shared xfce-minimal enlightenment kde5 lxde mate net-openrc pekwm-openrc sysmenu xfce-openbox-openrc gnome kde-minimal lxqt net openbox README.md xfce
Query Building
To query build an ISO (-q option), for example the xfce-openbox-openrc profile, the following command can be used:
$ buildiso -p xfce-openbox-openrc/ -b stable -q ==> manjaro-tools -> version: 0.9.6.4 -> user_config: /home/aaditya/.config/manjaro-tools.conf ==> PROFILE: -> buildsets: community|default|official|openrc -> buildset_iso: xfce-openbox-openrc/ -> is_buildset: false ==> OPTIONS: -> arch: x86_64 -> branch: stable -> chroots_iso: /mnt/datalinux2/manjaro-tools/buildiso ==> ARGS: -> clean_first: true -> clean_cache_xorg: true -> clean_cache_lng: true -> clean_cache_iso: true -> images_only: false -> iso_only: false ==> PATHS: -> sets_dir_iso: /etc/manjaro-tools/sets/iso -> cache_dir_iso: /mnt/datalinux2/manjaro-tools/cache/iso -> cache_dir_xorg: /mnt/datalinux2/manjaro-tools/cache/xorg/x86_64 -> cache_dir_lng: /mnt/datalinux2/manjaro-tools/cache/lng/x86_64 ==> IMAGES SETTINGS: -> manjaro_kernel: linux318 -> manjaro_version: 2015.03 -> plymouth_theme: manjaro-elegant ==> ISO SETTINGS: -> iso_label: MJRO0812 -> iso_version: openrc-2015-03-01 -> code_name: Ascella ==> LIVECD: -> username: manjaro -> password: manjaro -> addgroups: video,audio,power,disk,storage,optical,network,lp,scanner,wheel -> hostname: manjaro ==> BUILD QUEUE: --> Profile: [xfce-openbox-openrc/] ... -> Packages-Custom: Packages-Xfce -> initsys: openrc -> displaymanager: lxdm -> is_plymouth: false -> pacman_conf: pacman-multilib.conf -> work_dir: /mnt/datalinux2/manjaro-tools/buildiso/xfce-openbox-openrc//x86_64 -> iso_file: manjaro-xfce-openrc-2015-03-01-x86_64.iso -> start_openrc: syslog-ng cronie acpid alsasound dbus consolekit wicd bluetooth -> start_openrc_live: livecd mhwd-live pacman-init pacman-boot
Building
To actually build the ISO:
$ buildiso -p xfce-openbox-openrc/ -b stable ==> Start building [xfce-openbox-openrc/] --> Profile: [xfce-openbox-openrc/] ... -> Packages-Custom: Packages-Xfce ==> Cleaning up ... -> Deleting chroot 'root-image'... -> Deleting chroot 'xfce-image'... -> Cleaning [/mnt/datalinux2/manjaro-tools/cache/xorg/x86_64] ... -> Cleaning [/mnt/datalinux2/manjaro-tools/cache/lng/x86_64] ... --> Loading Packages: [Packages] ... ==> Prepare [Base installation] (root-image) ==> mkiso : Configuration Settings -> working directory: /mnt/datalinux2/manjaro-tools/buildiso/xfce-openbox-openrc//x86_64 -> image name: none -> pacman_mirrors: /usr/share/manjaro-tools/pacman-mirrors-stable.conf ==> Creating install root at /mnt/datalinux2/manjaro-tools/buildiso/xfce-openbox-openrc/x86_64/root-image ==> Installing packages to /mnt/datalinux2/manjaro-tools/buildiso/xfce-openbox-openrc/x86_64/root-image :: Synchronizing package databases... core 132.7 KiB 202K/s 00:01 [##############################################] 100% extra 1916.0 KiB 939K/s 00:02 [##############################################] 100% community 2.8 MiB 524K/s 00:05 [##############################################] 100% multilib 140.9 KiB 217K/s 00:01 [##############################################] 100% ... ... ... -> Cleaning up what we can warning: database file for 'core' does not exist warning: database file for 'extra' does not exist warning: database file for 'community' does not exist warning: database file for 'multilib' does not exist warning: database file for 'openrc-eudev' does not exist -> Copying overlay ... ==> Done [Base installation] (root-image) --> Loading Packages: [Packages-Xfce] ... ==> Prepare [xfce installation] (xfce-image) -> mount [root-image] on [xfce-image] ==> mkiso : Configuration Settings -> working directory: /mnt/datalinux2/manjaro-tools/buildiso/xfce-openbox-openrc//x86_64 -> image name: none -> pacman_mirrors: /usr/share/manjaro-tools/pacman-mirrors-stable.conf ==> Creating install root at /mnt/datalinux2/manjaro-tools/buildiso/xfce-openbox-openrc/x86_64/xfce-image ==> Installing packages to /mnt/datalinux2/manjaro-tools/buildiso/xfce-openbox-openrc/x86_64/xfce-image :: Synchronizing package databases... ... ... ... Generation complete. -> Cleaning up what we can warning: database file for 'core' does not exist warning: database file for 'extra' does not exist warning: database file for 'community' does not exist warning: database file for 'multilib' does not exist warning: database file for 'openrc-eudev' does not exist -> Copying xfce-overlay ... ==> Configuring [xfce-image] -> Configuring Displaymanager ... -> Configured: lxdm --> Congiguring OpenRC .... -> Setting syslog-ng ... -> Setting cronie ... -> Setting acpid ... -> Setting alsasound ... -> Setting dbus ... -> Setting consolekit ... -> Setting wicd ... -> Setting bluetooth ... --> Done configuring OpenRC ==> Done configuring [xfce-image] -> unmount xfce-image ==> Done [xfce installation] (xfce-image) --> Loading Packages: [Packages-Livecd] ... ==> Prepare [livecd-image] -> mount [livecd-image] on [xfce-image] -> append [root-image] on [livecd-image] ==> mkiso : Configuration Settings -> working directory: /mnt/datalinux2/manjaro-tools/buildiso/xfce-openbox-openrc//x86_64 -> image name: none -> pacman_mirrors: /usr/share/manjaro-tools/pacman-mirrors-testing.conf ==> Creating install root at /mnt/datalinux2/manjaro-tools/buildiso/xfce-openbox-openrc/x86_64/livecd-image ==> Installing packages to /mnt/datalinux2/manjaro-tools/buildiso/xfce-openbox-openrc/x86_64/livecd-image :: Synchronizing package databases... ... ... ... ==> Making bootable image -> Removing existing bootable image... ==> Creating ISO image... xorriso 1.3.8 : RockRidge filesystem manipulator, libburnia project. Drive current: -outdev 'stdio:/mnt/datalinux2/manjaro-tools/cache/iso/manjaro-xfce-openrc-2015-03-01-x86_64.iso' Media current: stdio file, overwriteable Media status : is blank Media summary: 0 sessions, 0 data blocks, 0 data, 62.7g free Added to ISO image: directory '/'='/mnt/datalinux2/manjaro-tools/buildiso/xfce-openbox-openrc/x86_64/iso' xorriso : UPDATE : 115 files added in 1 seconds xorriso : UPDATE : 115 files added in 1 seconds xorriso : NOTE : Copying to System Area: 432 bytes from file '/mnt/datalinux2/manjaro-tools/buildiso/xfce-openbox-openrc/x86_64/iso/isolinux/isohdpfx.bin' libisofs: NOTE : Aligned image size to cylinder size by 42 blocks xorriso : UPDATE : 2.75% done xorriso : UPDATE : 16.40% done xorriso : UPDATE : 27.90% done, estimate finish Sun Mar 01 23:10:20 2015 xorriso : UPDATE : 93.66% done xorriso : UPDATE : 99.20% done xorriso : UPDATE : 99.46% done xorriso : UPDATE : 99.47% done xorriso : UPDATE : 99.67% done xorriso : UPDATE : 99.68% done ISO image produced: 343552 sectors Written to medium : 343552 sectors at LBA 0 Writing to 'stdio:/mnt/datalinux2/manjaro-tools/cache/iso/manjaro-xfce-openrc-2015-03-01-x86_64.iso' completed successfully. ==> Done [Build ISO] ==> Creating [md5sum] ... -> md5sum: 78f4b335fc593b7df7edff46a4d91b5d manjaro-xfce-openrc-2015-03-01-x86_64.iso ==> Done [md5sum] --> Time compress_images: 9.43 minutes ==> Finished building [xfce-openbox-openrc/] --> Time make_profile: 13.77 minutes
Building with predownloaded Xorg packages
To build an ISO while retaining the previously downloaded cache of Xorg packages, the -x option can be used:
$ buildiso -p xfce-openbox-openrc/ -b stable -x
Building with a small config change
Supposing something only changed in config, like a setting, instead of building the whole ISO from scratch, the ISO build directory can be modified and the ISO can be rebuilt. For example, supposing one changed xfce-overlay/etc/skel/.conkyrc in the config, one can go into the work directory, modify the said file, and rebuild the ISO using the -cs option.
The work directory can be found using the -q option:
$ buildiso -p xfce-openbox-openrc/ -q ==> manjaro-tools -> version: 0.9.6.4 -> user_config: /home/aaditya/.config/manjaro-tools.conf ==> PROFILE: -> buildsets: community|default|official|openrc -> buildset_iso: xfce-openbox-openrc/ -> is_buildset: false ==> OPTIONS: -> arch: x86_64 -> branch: unstable -> chroots_iso: /mnt/datalinux2/manjaro-tools/buildiso ... ... ==> BUILD QUEUE: --> Profile: [xfce-openbox-openrc/] ... -> Packages-Custom: Packages-Xfce -> initsys: openrc -> displaymanager: lxdm -> is_plymouth: false -> pacman_conf: pacman-multilib.conf -> work_dir: /mnt/datalinux2/manjaro-tools/buildiso/xfce-openbox-openrc//x86_64 -> iso_file: manjaro-xfce-openrc-2015-03-01-x86_64.iso
(work directory highlighted in bold)
The required file can be edited as root, for example:
$ sudo nano /mnt/datalinux2/manjaro-tools/buildiso/xfce-openbox-openrc//x86_64/xfce-image/etc/skel/.conkyrc
Then the ISO can be rebuilt as:
$ buildiso -p xfce-openbox-openrc/ -cs
Alternative
First only the chroot for the ISO could be created, using the -i option, then the changes can be made, and finally an ISO can be built with the -sc option.
For example:
$ buildiso -p xfce-openrc/ -b stable -i ==> Start building [xfce-openrc/] --> Profile: [xfce-openrc/] -> Packages-Custom: Packages-Xfce ==> Cleaning up ... ... ... ==> Done [manjaro/iso/isolinux] ==> Creating [isomounts] -> Writing livecd entry ... -> Writing pkgs entry ... -> Writing xfce entry ... -> Writing root entry ... ==> Done creating [isomounts] --> Time build_images: 4.28 minutes ==> WARNING: Continue compress: buildiso -p xfce-openrc/ -sc ...
After this the changes in the work directory can be made, and the ISO can be generated with:
$ buildiso -p xfce-openrc/ -sc