| | |
| | | |
| | | This workspace contains examples of building open source software components using three methodologies. In the spec-files directory, the pkgbuild is use to build the components from a RPM spec file based build. In the components directory, a make(1) based build is used. |
| | | |
| | | spec-files |
| | | spec based build |
| | | |
| | | components |
| | | GNU make based build(s) |
| | | |
| | | make-rules |
| | | Shared rules to include in component specific makefiles (instructions) |
| | | |
| | | make-rules/configure.mk |
| | | shared rules for components using GNU auto* tools to build |
| | | |
| | | make-rules/ips.mk |
| | | shared rules for packaging under IPS |
| | | |
| | | make-rules/prep.mk |
| | | shared rules for downloading, unpacking, and patching component |
| | | source. |
| | | |
| | | make-rules/setup.py.mk |
| | | shared rules for components using python setup.py to build |
| | | |
| | | make-rules/shared-macros.mk |
| | | common macros that apply to building many things. |
| | | |
| | | tools |
| | | shared tools for building components |
| | | |
| | | transforms |
| | | shared pkgmogrify(1) transformations to apply to packages as they are |
| | | being built and published |
| | | Getting started with the Userland Consolidation |
| | | |
| | | |
| | | To build uing the GNU make based build you can |
| | | $ WS_TOP=$(pwd) ; cd components ; gmake -j 4 publish |
| | | This will construct a files based repository for publishing, create a log directory, build the components (4 at a time) and publish them to the repo. Logs of each component build will go in the logs directory |
| | | Getting Started |
| | | |
| | | This README provides a very brief overview of the gate, how to retrieve |
| | | a copy, and how to build it. Detailed documentation about the Userland |
| | | gate can be found in the 'doc' directory. Questions or comments about |
| | | the gate can be addressed to oi-dev@openindiana.org. |
| | | |
| | | Overview |
| | | |
| | | The Userland consolidation maintains a Git repository at |
| | | |
| | | https://github.com/OpenIndiana/oi-userland |
| | | |
| | | This gate contains build recipies, patches, IPS manifests, etc. necessary |
| | | to download, prep, build, test, package and publish open source software. |
| | | The build infrastructure is similiar to that of the SFW consolidation in |
| | | that it makes use of herarchical Makefiles which provide dependency and |
| | | recipe information for building the components. In order to build the |
| | | contents of the Userland gate, you need to clone it. Since you are |
| | | reading this, you probably already have. |
| | | |
| | | Getting the Bits |
| | | |
| | | As mentioned, the gate is stored in a Git repository. In order to |
| | | build or develop in the gate, you will need to clone it. You can do so |
| | | with the following command |
| | | |
| | | $ git clone https://github.com/OpenIndiana/oi-userland.git /scratch/clone |
| | | |
| | | This will create a replica of the various pieces that are checked into the |
| | | source code management system, but it does not retrieve the community |
| | | source archives associated with the gate content. To download the |
| | | community source associated with your cloned workspace, you will need to |
| | | execute the following: |
| | | |
| | | $ cd /scratch/clone/components |
| | | $ gmake download |
| | | |
| | | This will use GNU make and the downloading tool in the gate to walk through |
| | | all of the component directories downloading and validating the community |
| | | source archives from the gate machine or their canonical source repository. |
| | | |
| | | There are two variation to this that you may find interesting. First, you |
| | | can cause gmake(1) to perform it's work in parallel by adding '-j (jobs)' |
| | | to the command line. Second, if you are only interested in working on a |
| | | particular component, you can change directories to that component's |
| | | directory and use 'gmake download' from that to only get it's source |
| | | archive. |
| | | |
| | | Also, when you start to work with a new archive file - update the source |
| | | version in an existing recipe component, or start a new one from scratch - |
| | | you can use 'gmake fetch' to download the archive(s) defined in the new |
| | | recipe, calculate the checksums and *NOT* remove the archive because its |
| | | actual checksum does not match the value recorded in the recipe Makefile |
| | | (if any) so the download is deemed corrupted while you know it is not. |
| | | There is also a side-effect: by framework recipe, a file in the download |
| | | location only depends on the component recipe Makefile. So once an archive |
| | | is "fetched" (downloaded and not removed), it will not be re-verified - |
| | | the downloading script is just not called. This is a moderate problem, |
| | | since the "fetch" ability is a helper for recipe-makers doing initial |
| | | archive downloads in a certain situation, to save some traffic and time |
| | | on their workstations. You can still remove files fetched by a recipe |
| | | using 'gmake clobber'. |
| | | |
| | | Building the Bits |
| | | |
| | | You can build individual components or the contents of the entire gate. |
| | | |
| | | Integration with ccache to speed up re-builds |
| | | |
| | | If you happen to build the same sources several times (e.g. iterating |
| | | attempts to produce a working recipe, or maintaining an automated build |
| | | server), you can benefit from 'ccache' integration in 'oi-userland'. |
| | | Note that this feature is currently experimental and off by default. |
| | | |
| | | The 'ccache' component is available as part of the project repository. |
| | | Once you have the resulting package installed, you can pass the 'make' |
| | | argument or environment variable 'ENABLE_CCACHE=true' to wrap the GNU |
| | | compiler invocations with the caching program - so that the same inputs |
| | | would re-produce same outputs quickly. |
| | | |
| | | You can pre-set this variable in your user account '~/.profile' like this: |
| | | |
| | | ### To speed up OI-userland re-builds |
| | | ENABLE_CCACHE=true |
| | | export ENABLE_CCACHE |
| | | |
| | | Note: be wary of ccache's own CCACHE_DISABLE environment variable: any |
| | | value (empty, "false" etc.) is considered a "true" setting for ccache |
| | | booleans (and so disables the program, falling through to real compiler). |
| | | |
| | | Keeping all sources in one place |
| | | |
| | | The Userland consolidation tools automate download of required source |
| | | tarballs. By older default they were kept in each component's directory, |
| | | but you could centralize it by using the 'USERLAND_ARCHIVES' variable. |
| | | Recently the defaults change to pre-initialize 'USERLAND_ARCHIVES' to |
| | | point into '$(WS_TOP)/archives/' unless customized by the caller - for |
| | | example, to share the common download area between multiple workspaces. |
| | | |
| | | You can pre-set this variable in your user account '~/.profile' like |
| | | this (and note that the trailing slash is required): |
| | | |
| | | ### For oi-userland source files |
| | | USERLAND_ARCHIVES="$HOME/Downloads/" |
| | | export USERLAND_ARCHIVES |
| | | |
| | | See also the 'make-rules/shared-macros.mk' for 'INTERNAL_ARCHIVE_MIRROR', |
| | | 'EXTERNAL_ARCHIVE_MIRROR' and envvar 'DOWNLOAD_SEARCH_PATH' to get some |
| | | ideas about using HTTP mirrors to e.g. reduce network load and lags if you |
| | | can access a country- or organization-local mirror of opensource projects. |
| | | |
| | | Component build |
| | | |
| | | If you are only working on a single component, you can just build it using |
| | | following: |
| | | |
| | | setup the workspace for building components |
| | | |
| | | $ cd (your-workspace)/components ; gmake setup |
| | | |
| | | build the individual component |
| | | |
| | | $ cd (component-dir) ; gmake publish |
| | | |
| | | Complete Top Down build |
| | | |
| | | Complete top down builds are also possible by simply running |
| | | |
| | | $ cd (your-workspace)/components |
| | | $ gmake publish |
| | | |
| | | The 'publish' target will build each component and publish it to the |
| | | workspace IPS repo. |
| | | |
| | | Tools to help facilitate build zone creation will be integrated |
| | | shortly. If the zone you create to build your workspace in does not have |
| | | networking enabled, you can pre-download any community source archives into |
| | | your workspace from the global with: |
| | | |
| | | $ cd (your-workspace)/components |
| | | $ gmake download |
| | | |
| | | You can add parallelism to your builds by adding '-j (jobs)' to your gmake |
| | | command line arguments. Note that if the host is constrained on resources |
| | | or the component source Makefiles are poorly thought out, parallel builds |
| | | can fail - in this case subsequent single-job (sequential) builds should |
| | | succeed to complete the missing build products. |
| | | |
| | | It is worth noting that the OpenIndiana Hipster build server uses the |
| | | 'COMPONENT_BUILD_ARGS=-j4' option by default for moderate parallelization |
| | | of its builds. |
| | | |
| | | The gate should only incrementally build what it needs to based on what has |
| | | changed since you last built it. |