So according to Jenkins' antispam bot, the following is ... spam.
ehhh.... x.x
---
Jenkins setup:
```
Jenkins: 2.440.1
OS: Linux - 6.6.0-g076ede06c00a
Java: 23-ea - Debian (OpenJDK 64-Bit Server VM)
---
analysis-model-api:12.1.0
ant:497.v94e7d9fffa_b_9
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
authentication-tokens:1.53.v1c90fd9191a_b_
bootstrap5-api:5.3.2-4
bouncycastle-api:2.30.1.77-225.v26ea_c9455fd9
branch-api:2.1152.v6f101e97dd77
build-name-setter:2.4.1
build-timeout:1.32
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.0.2
cloudbees-folder:6.901.vb_4c7a_da_75da_3
command-launcher:107.v773860566e2e
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.11.0-95.v22a_d30ee5d36
conditional-buildstep:1.4.3
config-file-provider:968.ve1ca_eb_913f8c
configuration-as-code:1775.v810dc950b_514
credentials:1319.v7eb_51b_3a_c97b_
credentials-binding:657.v2b_19db_7d6e6d
dark-theme:416.v535839b_c4e88
dashboard-view:2.508.va_74654f026d1
data-tables-api:1.13.8-4
display-url-api:2.200.vb_9327d658781
docker-commons:439.va_3cb_0a_6a_fb_29
docker-workflow:572.v950f58993843
durable-task:550.v0930093c4b_a_6
echarts-api:5.4.3-4
email-ext:2.104
embeddable-build-status:467.v4a_954796e45d
font-awesome-api:6.5.1-3
forensics-api:2.4.0
git:5.2.1
git-client:4.6.0
git-parameter:0.9.19
git-server:114.v068a_c7cc2574
github:1.38.0
github-api:1.318-461.v7a_c09c9fa_d63
github-branch-source:1772.va_69eda_d018d4
gitlab-plugin:1.8.0
gradle:2.10
gson-api:2.10.1-15.v0d99f670e0a_7
instance-identity:185.v303dc7c645f9
ionicons-api:56.v1b_1c8c49374e
jackson2-api:2.16.1-373.ve709c6871598
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.9-1
jdk-tool:73.vddf737284550
jersey2-api:2.41-133.va_03323b_a_1396
jjwt-api:0.11.5-77.v646c772fddb_0
joda-time-api:2.12.7-29.v5a_b_e3a_82269a_
jquery3-api:3.7.1-2
json-api:20240205-27.va_007549e895c
json-path-api:2.9.0-33.v2527142f2e1d
junit:1259.v65ffcef24a_88
locale:431.v3435fa_8f8445
mailer:463.vedf8358e006b_
mapdb-api:1.0.9-28.vf251ce40855d
matrix-auth:3.2.1
matrix-project:822.824.v14451b_c0fd42
metrics:4.2.21-449.v6960d7c54c69
mina-sshd-api-common:2.12.0-90.v9f7fb_9fa_3d3b_
mina-sshd-api-core:2.12.0-90.v9f7fb_9fa_3d3b_
nodejs:1.6.1
okhttp-api:4.11.0-172.vda_da_1feeb_c6e
pam-auth:1.10
pipeline-build-step:540.vb_e8849e1a_b_d8
pipeline-github-lib:42.v0739460cda_c4
pipeline-graph-analysis:202.va_d268e64deb_3
pipeline-groovy-lib:704.vc58b_8890a_384
pipeline-input-step:491.vb_07d21da_1a_fb_
pipeline-milestone-step:111.v449306f708b_7
pipeline-model-api:2.2175.v76a_fff0a_2618
pipeline-model-definition:2.2175.v76a_fff0a_2618
pipeline-model-extensions:2.2175.v76a_fff0a_2618
pipeline-rest-api:2.34
pipeline-stage-step:305.ve96d0205c1c6
pipeline-stage-tags-metadata:2.2175.v76a_fff0a_2618
pipeline-stage-view:2.34
pipeline-utility-steps:2.16.2
plain-credentials:143.v1b_df8b_d3b_e48
plugin-util-api:4.1.0
prism-api:1.29.0-13
publish-over:0.22
resource-disposer:0.23
role-strategy:689.v731678c3e0eb_
run-condition:1.7
scm-api:683.vb_16722fb_b_80b_
script-security:1326.vdb_c154de8669
snakeyaml-api:2.2-111.vc6598e30cc65
ssh-agent:346.vda_a_c4f2c8e50
ssh-credentials:308.ve4497b_ccd8f4
ssh-slaves:2.948.vb_8050d697fec
sshd:3.322.v159e91f6a_550
structs:337.v1b_04ea_4df7c8
subversion:2.17.3
theme-manager:215.vc1ff18d67920
throttle-concurrents:2.14
timestamper:1.26
token-macro:400.v35420b_922dcb_
trilead-api:2.133.vfb_8a_7b_9c5dd1
variant:60.v7290fc0eb_b_cd
warnings-ng:11.0.0
workflow-aggregator:596.v8c21c963d92d
workflow-api:1291.v51fd2a_625da_7
workflow-basic-steps:1042.ve7b_140c4a_e0c
workflow-cps:3867.v535458ce43fd
workflow-durable-task-step:1331.vc8c2fed35334
workflow-job:1400.v7fd111b_ec82f
workflow-multibranch:773.vc4fe1378f1d5
workflow-scm-step:415.v434365564324
workflow-step-api:657.v03b_e8115821b_
workflow-support:865.v43e78cc44e0d
ws-cleanup:0.45
```
(I am limited to 2 links - apologies for the linkless links!)
Alright, with that done... Hello there!
I am new to Jenkins - but not to CI/CD. Before, I was trying out Laminar, Travis and others over the years. But now, I actually need CI/CD more than ever.
Here's the sitch: I have a VisionFive 2 (RISC-V SBC) that, whilst being [almost upstream](https://rvspace.org/en/project/JH7110_Upstream_Plan) still lacks a few very important things; two, to be precise. One being a variety of OS images, and another being Docker containers.
Since RISC-V is kinda "the new kid on the block" still, there is not that much in terms of OS images, kernel prebuilts and the likes. Well, I would like to change that. Sure, I could go to the Alpine mailing list and moan about the lack of an `alpine:3.19` image (which I might have to do still - but with way more substance behind it all) and hope for the best. But, I would rather not do so immediately, and instead get some work done that can be reused by others in the long-run.
So, I dropped OpenJDK and Jenkins on my VF2, wrote a SystemD unit and aside from TVHeadend and Home Assistant, there is nothing running on this box. Literally, I can actually see the bottom of htop! So there is a whole lot of unused compute that I would like to take advantage of - especially since I have a 1TB NVMe SSD in the box now.
And this is where Jenkins comes in - or, more precisely, a whole lot of pipelines. Basically, I have to build a few things in order to work my way to Alpine's `abuild` with a proper toolchain behind it, to then use those automations to produce packages, that I can then flush into a repository, which I can then use to programmatically create Alpine images - tagged ones, at that, with _their_ build instructions and dependency tracking.
So the structure is effectively this:
1. Build a Buildroot (buildroot/buildroot)
- I chose to pick ucLibc + gcc (c, c++) with Busybox and a few basics; keeping it simple.
2. `FROM scratch; ADD buildroot-sdk.tar.gz /` into a Docker image (let's tag that `buildroot:$git_tag`)
- ... And publish that to my Docker account so others can use it.
3. Tell Jenkins to build a new container `buildroot-abuild` that extends `buildroot` - but only with `abuild`.
4. Create a "dynamic" pipeline, that will start to feed package names from the abuild/aports repo into a pipeline in the sense of `for $pkg in $pkg_list { docker run --rm buildroot-abuild -v $abuild_repo:/abuild /* ... */ }`. The Alpine images for Docker are quite minimal, so I would start with those first. After each image finishes, there should be a couple of package files that can be collected and sent to an upstream repo (probably something like `scp $file user@host:$repo_path/ && ssh user@host rebuild-repo`)
5. Rinse and repeat untill the basics for `alpine:3.19` are satisfied.
6. Use the `alpine-make-rootfs` (alpinelinux/alpine-make-rootfs/tree/master) tool, apply a patch to change the location of `apk-static` and the repository URL and construct the new rootfs - then drop it into a container that we can savely tag `alpine:3.19` and publish!
The buildroot setup is quite easy; specify a repo, specify the config, specify the tag. Though I would like for Jenkins to automatically run new builds against new tags - treating them as releases, as long as they have no `-rcN` suffix (i.e. regEx test?).
Adding `abuild` into this is but a super simple Dockerfile away - however, this SHOULD always run after a new buildroot tag has been built, and should always use the latest version of `abuild` (tagged in the repo as a semver).
`abuild` runs need a working `buildroot-abuild` first and foremost - so once one of these exists and is live and available to be used, I want jenkins to get to work on building those one by one. What I do not know is: How would I actually tell it which packages to build, and which not to? Is there a script that can trigger something like a parameterized pipeline? I.e.: `jenkins-cli run pipeline buildroot-abuild --arg PKG=$pkg_name` would trigger Jenkins to start a job where it spins up an instance of `buildroot-abuild`, mounts the `abuilds` repo and then executes the build and collect/deploy the outputs?
Then, when a certain set of those `buildroot-abuild` builds have produced the neccessary outputs to construct the `alpine:3.19` rootfs, I want Jenkins to pick up on that and construct the container and publish it on my Docker profile.
So... I spent a lot of time planning this in the times I traveled to work and school and back. However, I am a little lost on:
- How do I organize those steps?
- How do I make pipeline B depend on A?
- How do I add parameters to a pipeline and make that part of a dependency? (`buildroot-abuild PKG=libwhatever` as a dependency to `alpine:3.19`, in this case)
- How do I "dynamically" provision agents (which the `buildroot-abuild` containers will be, after all)? Or do I have to wait untill that one is ready, then mark it as an agent and then continue setting up the rest?
I know that this is a lot of questions - and I tried to condense it and hopefuly I posted it into the right forum. I did look into Jenkinsfiles and thought of just making a monolithic repository containing an onslaught of those to describe the whole process.
The long-term goal is to not just build Alpine images, but possibly take advantage of the `buildroot` container to build OS images of other versions; Ubuntu, Debian, NixOS, ... Things, yknow? I have a kernel config, working on openSBI and U-Boot right now, so that I can supply those to create distro packages that can then become part of the built OS image. But - first, I need this Alpine image to make Podman shut up about missing layers. :D Because if I want to add this VF2 into a K3S cluster, I need it to have images it can work with. `alpine:edge` deviates strongly from the idea of a strictly described state - so I want to use a fixed release instead...and since the most common one is 3.19, that is the one I target.
Again, I apologize for this absolute **wall** but I hope I got both the forum and location as well as the structure of this right!
Thank you a lot for reading this whole thing, by the way. ^^
Kind regards,
Ingwie