| | |
| | | Getting started with the Userland Consolidation |
| | | |
| | | |
| | | Building the bits |
| | | Getting Started |
| | | |
| | | The Userland consolidation maintains a Mercurial gate at |
| | | |
| | | ssh://anon@hg.opensolaris.org//hg/userland/gate |
| | | |
| | | This README provides a very brief overview of the gate, how to retrieve |
| | | a copy, and how to build it. Detailed documentation about the Userland |
| | | gate can be found in the 'doc' directory. Questions or comments about |
| | | the gate can be addressed to oi-dev@openindiana.org. |
| | | |
| | | Overview |
| | | |
| | | The Userland consolidation maintains a Git repository at |
| | | |
| | | https://github.com/OpenIndiana/oi-userland |
| | | |
| | | This gate contains build recipies, patches, IPS manifests, etc. necessary |
| | | to download, prep, build, test, package and publish open source software. |
| | | In order to build the contents of the Userland gate, you need to clone it. |
| | | Since you are reading this, you probably already have, but in any event |
| | | you can do so with the following command |
| | | |
| | | $ hg clone ssh://anon@hg.opensolaris.org//hg/userland/gate /scratch/clone |
| | | |
| | | In order to build the bits either individually or collectively, you must |
| | | set the WS_TOP environment variable to point to the top of your workspace. |
| | | |
| | | $ export WS_TOP=/scratch/clone |
| | | |
| | | To build and publish the entire contents of the gate, you can use |
| | | |
| | | $ cd /scratch/clone |
| | | $ gmake publish |
| | | |
| | | To build and publish a specific component you need to initialize the |
| | | workspace by building the tools and creating a repository to publish |
| | | your results in. The easiest way to do this is to |
| | | |
| | | $ cd /scratch/clone |
| | | $ gmake setup |
| | | |
| | | Once you have initialize the the workspace, you can build individual |
| | | components by |
| | | |
| | | $ cd /scratch/clone/components/(component) |
| | | $ gmake publish |
| | | |
| | | All of the bits are are built will be published to the repository created |
| | | by the setup step (file:///scratch/clone/repo/) If you build the entire |
| | | contents of the gate, individual build logs for each component will be |
| | | located at /scratch/clone/logs/(target):(component).log |
| | | The build infrastructure is similiar to that of the SFW consolidation in |
| | | that it makes use of herarchical Makefiles which provide dependency and |
| | | recipe information for building the components. In order to build the |
| | | contents of the Userland gate, you need to clone it. Since you are |
| | | reading this, you probably already have. |
| | | |
| | | Getting the Bits |
| | | |
| | | As mentioned, the gate is stored in a Git repository. In order to |
| | | build or develop in the gate, you will need to clone it. You can do so |
| | | with the following command |
| | | |
| | | $ git clone https://github.com/OpenIndiana/oi-userland.git /scratch/clone |
| | | |
| | | This will create a replica of the various pieces that are checked into the |
| | | source code management system, but it does not retrieve the community |
| | | source archives associated with the gate content. To download the |
| | | community source associated with your cloned workspace, you will need to |
| | | execute the following: |
| | | |
| | | $ cd /scratch/clone/components |
| | | $ gmake download |
| | | |
| | | This will use GNU make and the downloading tool in the gate to walk through |
| | | all of the component directories downloading and validating the community |
| | | source archives from the gate machine or their canonical source repository. |
| | | |
| | | There are two variation to this that you may find interesting. First, you |
| | | can cause gmake(1) to perform it's work in parallel by adding '-j (jobs)' |
| | | to the command line. Second, if you are only interested in working on a |
| | | particular component, you can change directories to that component's |
| | | directory and use 'gmake download' from that to only get it's source |
| | | archive. |
| | | |
| | | Also, when you start to work with a new archive file - update the source |
| | | version in an existing recipe component, or start a new one from scratch - |
| | | you can use 'gmake fetch' to download the archive(s) defined in the new |
| | | recipe, calculate the checksums and *NOT* remove the archive because its |
| | | actual checksum does not match the value recorded in the recipe Makefile |
| | | (if any) so the download is deemed corrupted while you know it is not. |
| | | There is also a side-effect: by framework recipe, a file in the download |
| | | location only depends on the component recipe Makefile. So once an archive |
| | | is "fetched" (downloaded and not removed), it will not be re-verified - |
| | | the downloading script is just not called. This is a moderate problem, |
| | | since the "fetch" ability is a helper for recipe-makers doing initial |
| | | archive downloads in a certain situation, to save some traffic and time |
| | | on their workstations. You can still remove files fetched by a recipe |
| | | using 'gmake clobber'. |
| | | |
| | | Building the Bits |
| | | |
| | | You can build individual components or the contents of the entire gate. |
| | | |
| | | Integration with ccache to speed up re-builds |
| | | |
| | | If you happen to build the same sources several times (e.g. iterating |
| | | attempts to produce a working recipe, or maintaining an automated build |
| | | server), you can benefit from 'ccache' integration in 'oi-userland'. |
| | | Note that this feature is currently experimental and off by default. |
| | | |
| | | The 'ccache' component is available as part of the project repository. |
| | | Once you have the resulting package installed, you can pass the 'make' |
| | | argument or environment variable 'ENABLE_CCACHE=true' to wrap the GNU |
| | | compiler invocations with the caching program - so that the same inputs |
| | | would re-produce same outputs quickly. |
| | | |
| | | You can pre-set this variable in your user account '~/.profile' like this: |
| | | |
| | | ### To speed up OI-userland re-builds |
| | | ENABLE_CCACHE=true |
| | | export ENABLE_CCACHE |
| | | |
| | | Note: be wary of ccache's own CCACHE_DISABLE environment variable: any |
| | | value (empty, "false" etc.) is considered a "true" setting for ccache |
| | | booleans (and so disables the program, falling through to real compiler). |
| | | |
| | | Keeping all sources in one place |
| | | |
| | | The Userland consolidation tools automate download of required source |
| | | tarballs. By older default they were kept in each component's directory, |
| | | but you could centralize it by using the 'USERLAND_ARCHIVES' variable. |
| | | Recently the defaults change to pre-initialize 'USERLAND_ARCHIVES' to |
| | | point into '$(WS_TOP)/archives/' unless customized by the caller - for |
| | | example, to share the common download area between multiple workspaces. |
| | | |
| | | You can pre-set this variable in your user account '~/.profile' like |
| | | this (and note that the trailing slash is required): |
| | | |
| | | ### For oi-userland source files |
| | | USERLAND_ARCHIVES="$HOME/Downloads/" |
| | | export USERLAND_ARCHIVES |
| | | |
| | | See also the 'make-rules/shared-macros.mk' for 'INTERNAL_ARCHIVE_MIRROR', |
| | | 'EXTERNAL_ARCHIVE_MIRROR' and envvar 'DOWNLOAD_SEARCH_PATH' to get some |
| | | ideas about using HTTP mirrors to e.g. reduce network load and lags if you |
| | | can access a country- or organization-local mirror of opensource projects. |
| | | |
| | | Component build |
| | | |
| | | If you are only working on a single component, you can just build it using |
| | | following: |
| | | |
| | | setup the workspace for building components |
| | | |
| | | $ cd (your-workspace)/components ; gmake setup |
| | | |
| | | build the individual component |
| | | |
| | | $ cd (component-dir) ; gmake publish |
| | | |
| | | Complete Top Down build |
| | | |
| | | Complete top down builds are also possible by simply running |
| | | |
| | | $ cd (your-workspace)/components |
| | | $ gmake publish |
| | | |
| | | The 'publish' target will build each component and publish it to the |
| | | workspace IPS repo. |
| | | |
| | | Tools to help facilitate build zone creation will be integrated |
| | | shortly. If the zone you create to build your workspace in does not have |
| | | networking enabled, you can pre-download any community source archives into |
| | | your workspace from the global with: |
| | | |
| | | $ cd (your-workspace)/components |
| | | $ gmake download |
| | | |
| | | You can add parallelism to your builds by adding '-j (jobs)' to your gmake |
| | | command line arguments. Note that if the host is constrained on resources |
| | | or the component source Makefiles are poorly thought out, parallel builds |
| | | can fail - in this case subsequent single-job (sequential) builds should |
| | | succeed to complete the missing build products. |
| | | |
| | | It is worth noting that the OpenIndiana Hipster build server uses the |
| | | 'COMPONENT_BUILD_ARGS=-j4' option by default for moderate parallelization |
| | | of its builds. |
| | | |
| | | The gate should only incrementally build what it needs to based on what has |
| | | changed since you last built it. |