| commit | author | age | ||
| 9c75c0 | 1 | |
| 6514ee | 2 | Getting started with the Userland Consolidation |
| 9c75c0 | 3 | |
| NJ | 4 | |
| 3dc823 | 5 | Getting Started |
| NJ | 6 | |
| 7 | This README provides a very brief overview of the gate, how to retrieve | |
| 8 | a copy, and how to build it. Detailed documentation about the Userland | |
| 9 | gate can be found in the 'doc' directory. Questions or comments about | |
| 7b4f79 | 10 | the gate can be addressed to oi-dev@openindiana.org. |
| 3dc823 | 11 | |
| NJ | 12 | Overview |
| 6514ee | 13 | |
| ef4e01 | 14 | The Userland consolidation maintains a Git repository at |
| 3dc823 | 15 | |
| 7b4f79 | 16 | https://github.com/OpenIndiana/oi-userland |
| 3dc823 | 17 | |
| 6514ee | 18 | This gate contains build recipies, patches, IPS manifests, etc. necessary |
| NJ | 19 | to download, prep, build, test, package and publish open source software. |
| 3dc823 | 20 | The build infrastructure is similiar to that of the SFW consolidation in |
| NJ | 21 | that it makes use of herarchical Makefiles which provide dependency and |
| 22 | recipe information for building the components. In order to build the | |
| 23 | contents of the Userland gate, you need to clone it. Since you are | |
| 24 | reading this, you probably already have. | |
| 6514ee | 25 | |
| 3dc823 | 26 | Getting the Bits |
| 9c75c0 | 27 | |
| ef4e01 | 28 | As mentioned, the gate is stored in a Git repository. In order to |
| 3dc823 | 29 | build or develop in the gate, you will need to clone it. You can do so |
| NJ | 30 | with the following command |
| 7b4f79 | 31 | |
| ef4e01 | 32 | $ git clone https://github.com/OpenIndiana/oi-userland.git /scratch/clone |
| 3dc823 | 33 | |
| NJ | 34 | This will create a replica of the various pieces that are checked into the |
| 35 | source code management system, but it does not retrieve the community | |
| 36 | source archives associated with the gate content. To download the | |
| 37 | community source associated with your cloned workspace, you will need to | |
| 38 | execute the following: | |
| 39 | ||
| 40 | $ cd /scratch/clone/components | |
| 41 | $ gmake download | |
| 42 | ||
| 43 | This will use GNU make and the downloading tool in the gate to walk through | |
| 44 | all of the component directories downloading and validating the community | |
| 45 | source archives from the gate machine or their canonical source repository. | |
| 46 | ||
| 47 | There are two variation to this that you may find interesting. First, you | |
| 48 | can cause gmake(1) to perform it's work in parallel by adding '-j (jobs)' | |
| 49 | to the command line. Second, if you are only interested in working on a | |
| 50 | particular component, you can change directories to that component's | |
| 51 | directory and use 'gmake download' from that to only get it's source | |
| 52 | archive. | |
| 53 | ||
| 2119e2 | 54 | Also, when you start to work with a new archive file - update the source |
| JK | 55 | version in an existing recipe component, or start a new one from scratch - |
| 56 | you can use 'gmake fetch' to download the archive(s) defined in the new | |
| 57 | recipe, calculate the checksums and *NOT* remove the archive because its | |
| 58 | actual checksum does not match the value recorded in the recipe Makefile | |
| 59 | (if any) so the download is deemed corrupted while you know it is not. | |
| 60 | There is also a side-effect: by framework recipe, a file in the download | |
| 61 | location only depends on the component recipe Makefile. So once an archive | |
| 62 | is "fetched" (downloaded and not removed), it will not be re-verified - | |
| 63 | the downloading script is just not called. This is a moderate problem, | |
| 64 | since the "fetch" ability is a helper for recipe-makers doing initial | |
| 65 | archive downloads in a certain situation, to save some traffic and time | |
| 66 | on their workstations. You can still remove files fetched by a recipe | |
| 67 | using 'gmake clobber'. | |
| 68 | ||
| 7b4f79 | 69 | Building the Bits |
| 3dc823 | 70 | |
| NJ | 71 | You can build individual components or the contents of the entire gate. |
| 7b4f79 | 72 | |
| c5572a | 73 | Integration with ccache to speed up re-builds |
| JK | 74 | |
| 75 | If you happen to build the same sources several times (e.g. iterating | |
| 76 | attempts to produce a working recipe, or maintaining an automated build | |
| 77 | server), you can benefit from 'ccache' integration in 'oi-userland'. | |
| 78 | Note that this feature is currently experimental and off by default. | |
| 79 | ||
| 80 | The 'ccache' component is available as part of the project repository. | |
| 81 | Once you have the resulting package installed, you can pass the 'make' | |
| 82 | argument or environment variable 'ENABLE_CCACHE=true' to wrap the GNU | |
| 83 | compiler invocations with the caching program - so that the same inputs | |
| 84 | would re-produce same outputs quickly. | |
| 85 | ||
| 86 | You can pre-set this variable in your user account '~/.profile' like this: | |
| 87 | ||
| 88 | ### To speed up OI-userland re-builds | |
| 89 | ENABLE_CCACHE=true | |
| 90 | export ENABLE_CCACHE | |
| 91 | ||
| fcf34c | 92 | Note: be wary of ccache's own CCACHE_DISABLE environment variable: any |
| JK | 93 | value (empty, "false" etc.) is considered a "true" setting for ccache |
| 94 | booleans (and so disables the program, falling through to real compiler). | |
| 95 | ||
| 7b4f79 | 96 | Keeping all sources in one place |
| JK | 97 | |
| 98 | The Userland consolidation tools automate download of required source | |
| fe4a1d | 99 | tarballs. By older default they were kept in each component's directory, |
| JK | 100 | but you could centralize it by using the 'USERLAND_ARCHIVES' variable. |
| 101 | Recently the defaults change to pre-initialize 'USERLAND_ARCHIVES' to | |
| 102 | point into '$(WS_TOP)/archives/' unless customized by the caller - for | |
| 103 | example, to share the common download area between multiple workspaces. | |
| 7b4f79 | 104 | |
| c5572a | 105 | You can pre-set this variable in your user account '~/.profile' like |
| 7b4f79 | 106 | this (and note that the trailing slash is required): |
| JK | 107 | |
| c5572a | 108 | ### For oi-userland source files |
| JK | 109 | USERLAND_ARCHIVES="$HOME/Downloads/" |
| 110 | export USERLAND_ARCHIVES | |
| 3dc823 | 111 | |
| fe4a1d | 112 | See also the 'make-rules/shared-macros.mk' for 'INTERNAL_ARCHIVE_MIRROR', |
| JK | 113 | 'EXTERNAL_ARCHIVE_MIRROR' and envvar 'DOWNLOAD_SEARCH_PATH' to get some |
| 114 | ideas about using HTTP mirrors to e.g. reduce network load and lags if you | |
| 115 | can access a country- or organization-local mirror of opensource projects. | |
| 116 | ||
| 3dc823 | 117 | Component build |
| NJ | 118 | |
| 119 | If you are only working on a single component, you can just build it using | |
| 120 | following: | |
| 121 | ||
| 122 | setup the workspace for building components | |
| 123 | ||
| 9dfd18 | 124 | $ cd (your-workspace)/components ; gmake setup |
| 3dc823 | 125 | |
| NJ | 126 | build the individual component |
| 127 | ||
| 128 | $ cd (component-dir) ; gmake publish | |
| 129 | ||
| 7b4f79 | 130 | Complete Top Down build |
| 3dc823 | 131 | |
| NJ | 132 | Complete top down builds are also possible by simply running |
| 133 | ||
| 9dfd18 | 134 | $ cd (your-workspace)/components |
| NJ | 135 | $ gmake publish |
| 3dc823 | 136 | |
| 9dfd18 | 137 | The 'publish' target will build each component and publish it to the |
| NJ | 138 | workspace IPS repo. |
| 7b4f79 | 139 | |
| 9dfd18 | 140 | Tools to help facilitate build zone creation will be integrated |
| 3dc823 | 141 | shortly. If the zone you create to build your workspace in does not have |
| NJ | 142 | networking enabled, you can pre-download any community source archives into |
| 143 | your workspace from the global with: | |
| 144 | ||
| 9dfd18 | 145 | $ cd (your-workspace)/components |
| 3dc823 | 146 | $ gmake download |
| NJ | 147 | |
| 7b4f79 | 148 | You can add parallelism to your builds by adding '-j (jobs)' to your gmake |
| JK | 149 | command line arguments. Note that if the host is constrained on resources |
| 150 | or the component source Makefiles are poorly thought out, parallel builds | |
| 151 | can fail - in this case subsequent single-job (sequential) builds should | |
| 152 | succeed to complete the missing build products. | |
| 3dc823 | 153 | |
| 7b4f79 | 154 | It is worth noting that the OpenIndiana Hipster build server uses the |
| c5572a | 155 | 'COMPONENT_BUILD_ARGS=-j4' option by default for moderate parallelization |
| 7b4f79 | 156 | of its builds. |
| JK | 157 | |
| 158 | The gate should only incrementally build what it needs to based on what has | |
| 159 | changed since you last built it. | |
