commit | author | age
|
9c75c0
|
1 |
|
6514ee
|
2 |
Getting started with the Userland Consolidation |
9c75c0
|
3 |
|
NJ |
4 |
|
3dc823
|
5 |
Getting Started |
NJ |
6 |
|
|
7 |
This README provides a very brief overview of the gate, how to retrieve |
|
8 |
a copy, and how to build it. Detailed documentation about the Userland |
|
9 |
gate can be found in the 'doc' directory. Questions or comments about |
7b4f79
|
10 |
the gate can be addressed to oi-dev@openindiana.org. |
3dc823
|
11 |
|
NJ |
12 |
Overview |
6514ee
|
13 |
|
ef4e01
|
14 |
The Userland consolidation maintains a Git repository at |
3dc823
|
15 |
|
7b4f79
|
16 |
https://github.com/OpenIndiana/oi-userland |
3dc823
|
17 |
|
6514ee
|
18 |
This gate contains build recipies, patches, IPS manifests, etc. necessary |
NJ |
19 |
to download, prep, build, test, package and publish open source software. |
3dc823
|
20 |
The build infrastructure is similiar to that of the SFW consolidation in |
NJ |
21 |
that it makes use of herarchical Makefiles which provide dependency and |
|
22 |
recipe information for building the components. In order to build the |
|
23 |
contents of the Userland gate, you need to clone it. Since you are |
|
24 |
reading this, you probably already have. |
6514ee
|
25 |
|
3dc823
|
26 |
Getting the Bits |
9c75c0
|
27 |
|
ef4e01
|
28 |
As mentioned, the gate is stored in a Git repository. In order to |
3dc823
|
29 |
build or develop in the gate, you will need to clone it. You can do so |
NJ |
30 |
with the following command |
7b4f79
|
31 |
|
ef4e01
|
32 |
$ git clone https://github.com/OpenIndiana/oi-userland.git /scratch/clone |
3dc823
|
33 |
|
NJ |
34 |
This will create a replica of the various pieces that are checked into the |
|
35 |
source code management system, but it does not retrieve the community |
|
36 |
source archives associated with the gate content. To download the |
|
37 |
community source associated with your cloned workspace, you will need to |
|
38 |
execute the following: |
|
39 |
|
|
40 |
$ cd /scratch/clone/components |
|
41 |
$ gmake download |
|
42 |
|
|
43 |
This will use GNU make and the downloading tool in the gate to walk through |
|
44 |
all of the component directories downloading and validating the community |
|
45 |
source archives from the gate machine or their canonical source repository. |
|
46 |
|
|
47 |
There are two variation to this that you may find interesting. First, you |
|
48 |
can cause gmake(1) to perform it's work in parallel by adding '-j (jobs)' |
|
49 |
to the command line. Second, if you are only interested in working on a |
|
50 |
particular component, you can change directories to that component's |
|
51 |
directory and use 'gmake download' from that to only get it's source |
|
52 |
archive. |
|
53 |
|
2119e2
|
54 |
Also, when you start to work with a new archive file - update the source |
JK |
55 |
version in an existing recipe component, or start a new one from scratch - |
|
56 |
you can use 'gmake fetch' to download the archive(s) defined in the new |
|
57 |
recipe, calculate the checksums and *NOT* remove the archive because its |
|
58 |
actual checksum does not match the value recorded in the recipe Makefile |
|
59 |
(if any) so the download is deemed corrupted while you know it is not. |
|
60 |
There is also a side-effect: by framework recipe, a file in the download |
|
61 |
location only depends on the component recipe Makefile. So once an archive |
|
62 |
is "fetched" (downloaded and not removed), it will not be re-verified - |
|
63 |
the downloading script is just not called. This is a moderate problem, |
|
64 |
since the "fetch" ability is a helper for recipe-makers doing initial |
|
65 |
archive downloads in a certain situation, to save some traffic and time |
|
66 |
on their workstations. You can still remove files fetched by a recipe |
|
67 |
using 'gmake clobber'. |
|
68 |
|
7b4f79
|
69 |
Building the Bits |
3dc823
|
70 |
|
NJ |
71 |
You can build individual components or the contents of the entire gate. |
7b4f79
|
72 |
|
c5572a
|
73 |
Integration with ccache to speed up re-builds |
JK |
74 |
|
|
75 |
If you happen to build the same sources several times (e.g. iterating |
|
76 |
attempts to produce a working recipe, or maintaining an automated build |
|
77 |
server), you can benefit from 'ccache' integration in 'oi-userland'. |
|
78 |
Note that this feature is currently experimental and off by default. |
|
79 |
|
|
80 |
The 'ccache' component is available as part of the project repository. |
|
81 |
Once you have the resulting package installed, you can pass the 'make' |
|
82 |
argument or environment variable 'ENABLE_CCACHE=true' to wrap the GNU |
|
83 |
compiler invocations with the caching program - so that the same inputs |
|
84 |
would re-produce same outputs quickly. |
|
85 |
|
|
86 |
You can pre-set this variable in your user account '~/.profile' like this: |
|
87 |
|
|
88 |
### To speed up OI-userland re-builds |
|
89 |
ENABLE_CCACHE=true |
|
90 |
export ENABLE_CCACHE |
|
91 |
|
fcf34c
|
92 |
Note: be wary of ccache's own CCACHE_DISABLE environment variable: any |
JK |
93 |
value (empty, "false" etc.) is considered a "true" setting for ccache |
|
94 |
booleans (and so disables the program, falling through to real compiler). |
|
95 |
|
7b4f79
|
96 |
Keeping all sources in one place |
JK |
97 |
|
|
98 |
The Userland consolidation tools automate download of required source |
fe4a1d
|
99 |
tarballs. By older default they were kept in each component's directory, |
JK |
100 |
but you could centralize it by using the 'USERLAND_ARCHIVES' variable. |
|
101 |
Recently the defaults change to pre-initialize 'USERLAND_ARCHIVES' to |
|
102 |
point into '$(WS_TOP)/archives/' unless customized by the caller - for |
|
103 |
example, to share the common download area between multiple workspaces. |
7b4f79
|
104 |
|
c5572a
|
105 |
You can pre-set this variable in your user account '~/.profile' like |
7b4f79
|
106 |
this (and note that the trailing slash is required): |
JK |
107 |
|
c5572a
|
108 |
### For oi-userland source files |
JK |
109 |
USERLAND_ARCHIVES="$HOME/Downloads/" |
|
110 |
export USERLAND_ARCHIVES |
3dc823
|
111 |
|
fe4a1d
|
112 |
See also the 'make-rules/shared-macros.mk' for 'INTERNAL_ARCHIVE_MIRROR', |
JK |
113 |
'EXTERNAL_ARCHIVE_MIRROR' and envvar 'DOWNLOAD_SEARCH_PATH' to get some |
|
114 |
ideas about using HTTP mirrors to e.g. reduce network load and lags if you |
|
115 |
can access a country- or organization-local mirror of opensource projects. |
|
116 |
|
3dc823
|
117 |
Component build |
NJ |
118 |
|
|
119 |
If you are only working on a single component, you can just build it using |
|
120 |
following: |
|
121 |
|
|
122 |
setup the workspace for building components |
|
123 |
|
9dfd18
|
124 |
$ cd (your-workspace)/components ; gmake setup |
3dc823
|
125 |
|
NJ |
126 |
build the individual component |
|
127 |
|
|
128 |
$ cd (component-dir) ; gmake publish |
|
129 |
|
7b4f79
|
130 |
Complete Top Down build |
3dc823
|
131 |
|
NJ |
132 |
Complete top down builds are also possible by simply running |
|
133 |
|
9dfd18
|
134 |
$ cd (your-workspace)/components |
NJ |
135 |
$ gmake publish |
3dc823
|
136 |
|
9dfd18
|
137 |
The 'publish' target will build each component and publish it to the |
NJ |
138 |
workspace IPS repo. |
7b4f79
|
139 |
|
9dfd18
|
140 |
Tools to help facilitate build zone creation will be integrated |
3dc823
|
141 |
shortly. If the zone you create to build your workspace in does not have |
NJ |
142 |
networking enabled, you can pre-download any community source archives into |
|
143 |
your workspace from the global with: |
|
144 |
|
9dfd18
|
145 |
$ cd (your-workspace)/components |
3dc823
|
146 |
$ gmake download |
NJ |
147 |
|
7b4f79
|
148 |
You can add parallelism to your builds by adding '-j (jobs)' to your gmake |
JK |
149 |
command line arguments. Note that if the host is constrained on resources |
|
150 |
or the component source Makefiles are poorly thought out, parallel builds |
|
151 |
can fail - in this case subsequent single-job (sequential) builds should |
|
152 |
succeed to complete the missing build products. |
3dc823
|
153 |
|
7b4f79
|
154 |
It is worth noting that the OpenIndiana Hipster build server uses the |
c5572a
|
155 |
'COMPONENT_BUILD_ARGS=-j4' option by default for moderate parallelization |
7b4f79
|
156 |
of its builds. |
JK |
157 |
|
|
158 |
The gate should only incrementally build what it needs to based on what has |
|
159 |
changed since you last built it. |