aboutsummaryrefslogtreecommitdiffstats
path: root/.gitlab-ci.yml
Commit message (Collapse)AuthorAgeFilesLines
* CI: reduce bandwidth for git pullBenjamin Tissoires2020-07-031-1/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Over the last 7 days, git pulls represented a total of 1.7 TB. On those 1.7 TB, we can see: - ~300 GB for the CI farm on hetzner - ~730 GB for the CI farm on packet.net - ~680 GB for the rest of the world We can not really change the rest of the world*, but we can certainly reduce the egress costs towards our CI farms. Right now, the gitlab runners are not doing a good job at caching the git trees for the various jobs we make, and we end up with a lot of cache-misses. A typical pipeline ends up with a good 2.8GB of git pull data. (a compressed archive of the mesa folder accounts for 280MB) In this patch, we implemented what was suggested in https://gitlab.com/gitlab-org/gitlab/-/issues/215591#note_334642576 - we host a brand new MinIO server on packet - jobs can upload files on 2 locations: * git-cache/<namespace>/<project>/<branch-name>.tar.gz * artifacts/<namespace>/<project>/<pipeline-id>/ - the authorization is handled by gitlab with short tokens valid only for the time of the job is running - whenever a job runs, the runner are configured to execute (eval) $CI_PRE_CLONE_SCRIPT - this variable is set globally to download the current cache from the MinIO packet server, unpack it and replace the possibly out of date cache found on the runner - then git fetch is run by the runner, and only the delta between the upstream tree and the local tree gets pulled. We can rebuild the git cache in a schedule job (once a day seems sufficient), and then we can stop the cache miss entirely. First results showed that instead of pulling 280MB of data in my fork, I got a pull of only 250KB. That should help us. * arguably, there are other farms in the rest of the world, so hopefully we can change those too. Reviewed-by: Michel Dänzer <[email protected]> Reviewed-by: Peter Hutterer <[email protected]> Signed-off-by: Benjamin Tissoires <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5428>
* ci: Move deploy stage between container & build stagesMichel Dänzer2020-07-021-1/+1
| | | | | | | | | | Having it as the last stage meant that the pages job could only run once all other jobs had finished. The new position means it can run in parallel with build jobs. Reviewed-by: Erik Faye-Lund <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5711>
* ci: Use "when: always" for pages jobMichel Dänzer2020-07-021-1/+1
| | | | | | | | | | | | | "when: on_success" meant that that the job wouldn't run automatically until all jobs of all earlier stages passed, and would be skipped if any of them failed. But we need to always run this job if any documentation files were modified. Fixes: 8e2cb8ef276b "gitlab-ci: Extend .ci-run-policy template for docs jobs" Reviewed-by: Erik Faye-Lund <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5711>
* CI: Disable Windows build due to unstable infrastructureDaniel Stone2020-07-021-2/+2
| | | | | | | | | The Windows runner is having a lot of issues cloning repositories, failing early due to an unhandled HTTP error. Temporarily disable it until we can figure out what's going on and fix it. Signed-off-by: Daniel Stone <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5716>
* gitlab-ci: Do not create the "success" job when the test-docs job existsMichel Dänzer2020-07-011-5/+9
| | | | | | | | | | It's redundant in that case. v2: * Adapt to v2 of test-docs job rules. Reviewed-by: Daniel Stone <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
* gitlab-ci: Extend .ci-run-policy template for docs jobsMichel Dänzer2020-07-011-9/+9
| | | | | | | | | | | | Requires using rules: in the pages job as well, so it doesn't inherit the rules from the template. v2: * Add comment explaining that cases not covered by explicit rules default to "when: never". Reviewed-by: Daniel Stone <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
* gitlab-ci: Use rules: instead of except:/only: for test-docs jobMichel Dänzer2020-07-011-7/+11
| | | | | | | | | | | | | | Only run the job automatically for Marge Bot, otherwise let it be triggered manually. v2: * Never run this job for the main project, since it's only needed in pre-merge pipelines. * Add comment explaining that cases not covered by explicit rules default to "when: never". Reviewed-by: Daniel Stone <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
* ci: move test-docs to container stageErik Faye-Lund2020-07-011-4/+5
| | | | | | | | While we're at it, rename it to reflect that we're now also testing docs here. Reviewed-by: Michel Dänzer <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
* ci: move deploy-stage later in the pipelineErik Faye-Lund2020-07-011-1/+1
| | | | | | | This makes it not clutter up the pipeline results page so much. Reviewed-by: Michel Dänzer <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
* ci: test docs for non-master buildsErik Faye-Lund2020-07-011-1/+14
| | | | | | | This ensures that we test on CI before merge-requests gets merged. Reviewed-by: Michel Dänzer <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
* ci: only build docs if any docs changedErik Faye-Lund2020-07-011-1/+5
| | | | | | Reviewed-by: Michel Dänzer <[email protected]> Reviewed-by: Daniel Stone <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
* ci: only build docs in the upstream-repoErik Faye-Lund2020-07-011-1/+1
| | | | | | Reviewed-by: Michel Dänzer <[email protected]> Reviewed-by: Daniel Stone <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
* gitlab-ci: Automatically run pipelines for Marge Bot pre-merge onlyMichel Dänzer2020-07-011-7/+14
| | | | | | | | | | | | | Marge only merges an MR if the pipeline passed. Running the pipeline again after merging is redundant. v2: * Add rule to ensure docker images are up to date in the main project registry (Eric Anholt) Reviewed-by: Eric Anholt <[email protected]> Acked-by: Daniel Stone <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5680>
* CI: Re-enable the Windows VS2019 build jobDaniel Stone2020-06-301-1/+1
| | | | | | | | | | Let's try this and see how it goes. Signed-off-by: Daniel Stone <[email protected]> Reviewed-by: Michel Dänzer <[email protected]> Reviewed-by: Eric Engestrom <[email protected]> Reviewed-by: Eric Anholt <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5689>
* ci/baremetal: Bump the kernel to a recent drm-msm-fixes for msm semaphores.Eric Anholt2020-06-261-1/+1
| | | | | | We need this to test the new VK feature we're about to land. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2769>
* gitlab-ci: attach the Fossilize log file as artifact on failureSamuel Pitoiset2020-06-261-0/+5
| | | | | | | | It might be help. Signed-off-by: Samuel Pitoiset <[email protected]> Reviewed-by: Eric Anholt <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5627>
* ci: Remove a stray "always" on the freedreno traces job.Eric Anholt2020-06-251-2/+0
| | | | | | | | | This was making it so that the CI would error if the set of files modified or the pipeline involvd meant the jobs we depend on weren't enabled. It was just some misplaced debug leftovers of mine. Fixes: b88c46fa11ae ("ci: Add a freedreno a630 tracie run.") Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5653>
* ci: Add a freedreno a630 tracie run.Eric Anholt2020-06-251-0/+14
| | | | | | | | | This job runs in about one minute on the current set of traces, and has successfully revealed some bugs in our current rendering. Takes about 7 minutes currently. Reviewed-by: Tomeu Vizoso <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5433>
* ci: Include trace replay support in ARM rootfses.Rohan Garg2020-06-251-3/+3
| | | | | | | | | | | | | | Builds the renderdoc and apitrace programs so we can replay GL traces on DUTs. [Separated out from 5472's commit that also enabled the jobs in LAVA, dropped unnecessary python packages from arm_build, fixed up arm64_test build, traces-db in baremetal, new commit message by anholt] Signed-off-by: Rohan Garg <[email protected]> Signed-off-by: Tomeu Vizoso <[email protected]> Reviewed-by: Eric Anholt <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5433>
* ci/bare-metal: Don't include dev packages in arm*test.Eric Anholt2020-06-251-1/+1
| | | | | | | We just need these to build our rootfs, clean them out afterwards. Reviewed-by: Tomeu Vizoso <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5433>
* ci: Move ARM rootfses to stableTomeu Vizoso2020-06-251-3/+3
| | | | | | | | | | | | | We build in Debian buster but were currently testing in bullseye-based ramdisks. This has started being a problem since Python 3.7 was removed from bullseye. [ Also bumped arm_test containers, by anholt ] Signed-off-by: Tomeu Vizoso <[email protected]> Acked-by: Alyssa Rosenzweig <[email protected]> Reviewed-by: Eric Anholt <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5433>
* radv: enable ACO by defaultDaniel Schürmann2020-06-251-1/+0
| | | | | | | No more dragons have been seen, caution is still required... Reviewed-by: Samuel Pitoiset <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5445>
* gitlab-ci: stop testing RADV with LLVMSamuel Pitoiset2020-06-231-6/+1
| | | | | | | | | ACO is going to be our default compiler soon and it seems useless to waste CI resources for LLVM. Signed-off-by: Samuel Pitoiset <[email protected]> Acked-by: Bas Nieuwenhuizen <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5543>
* gitlab-ci: Update CTS runnerTomeu Vizoso2020-06-231-1/+1
| | | | | | | | | We need a newer version to be able to successfully run the OpenGL suites in dEQP. Signed-off-by: Tomeu Vizoso <[email protected]> Reviewed-by: Eric Anholt <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5494>
* gitlab-ci: Test virgl with Khronos' OpenGL CTSTomeu Vizoso2020-06-231-1/+16
| | | | | | Signed-off-by: Tomeu Vizoso <[email protected]> Reviewed-by: Eric Anholt <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5494>
* gitlab-ci: Add manual tests for Virgl using GLES on the hostTomeu Vizoso2020-06-231-9/+32
| | | | | | | | The ones that run automatically will use big GL on the host. Signed-off-by: Tomeu Vizoso <[email protected]> Reviewed-by: Eric Anholt <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5494>
* gitlab-ci: Run more dEQP tests for virglTomeu Vizoso2020-06-231-4/+2
| | | | | | | | | Llvmpipe seems to have become faster, and we can run more tests while still being under 5 minutes per job. Signed-off-by: Tomeu Vizoso <[email protected]> Reviewed-by: Eric Anholt <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5494>
* gitlab-ci: drop gettext from the build imagesEric Engestrom2020-06-221-3/+3
| | | | | | Suggested-by: Pierre-Eric Pelloux-Prayer <[email protected]> Signed-off-by: Eric Engestrom <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5440>
* ci: Enable NIR validation on a630 GLES2 and VK tests.Eric Anholt2020-06-191-1/+4
| | | | | | | | We get through GLES2 in 5.5 minutes and the vk subset in 8 minutes, so we can spare the CPU time on these tests. Acked-by: Rob Clark <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5554>
* ci: Bump vulkan CTS to 1.2.3.0.Eric Anholt2020-06-191-2/+2
| | | | | | | | | | | | | | Looks like it fixes some potentially important VK test bugs. But also, it fixes the GLES31 SSBO layout tests to not be so excessively large, so we can run them in a reasonable time now. Note that a630 fail list is reset, since the test list has changed and so we end up with a different subset of tests being run. Interestingly, in the process the semaphore tests are now reporting "NotSupported (Exporting and importing semaphore type not supported at vktSynchronizationSignalOrderTests.cpp:513)" where they weren't before. Reviewed-by: Samuel Pitoiset <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5554>
* ci: disable the windows tests until the runner can be stabilized againEric Anholt2020-06-181-1/+1
| | | | | | | | | | | I've been getting a stream of errors from Marge today like: FAILED: src/gallium/drivers/llvmpipe/lp_test_conv.exe link @src/gallium/drivers/llvmpipe/lp_test_conv.exe.rsp LINK : fatal error LNK1102: out of memory [1080/1141] Compiling C object src/gallium/frontends/wgl/945cc3d@@wgl@sta/stw_getprocaddress.c.obj Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5534>
* gitlab-ci: build and deploy docsErik Faye-Lund2020-06-131-0/+13
| | | | | | | | | Dunno if alpine is a good idea. It's what the gitlab docs use for most of their examples, so that's what I've gone with... Can probably be changed to something else if wanted. Reviewed-by: Eric Engestrom <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4630>
* ci/bare-metal: Stop fetching the git tree.Eric Anholt2020-06-121-3/+4
| | | | | | | | | | | | | | Like for LAVA, make the tradeoff of moving the test scripts and data (55k) into the artifacts in order to make the per-build jobs not have to pull down the git tree (hundreds of MB when you don't hit a cached container for your specific user, which I see happen multiple times a day in my CI runs). To do this, we have to be a bit more careful in some places about our working directory potentially being dirty. Reviewed-by: Christian Gmeiner <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5393>
* ci/bare-metal: Use the deqp-runner bits straight out of the artifacts.Eric Anholt2020-06-121-3/+6
| | | | | | | | | | | | We've already uploaded and downloaded them from fd.o and put them in the rootfs, so we can clean up the extra prep work. Our test job now extends from .test so that the artifacts' install dir with all the scripts is extracted. This required moving the dependency on meson-testing to the x86 test-gl/test-vk job blocks. Reviewed-by: Christian Gmeiner <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5393>
* ci/bare-metal: Make which test to run configurable.Eric Anholt2020-06-121-0/+1
| | | | | | | I'll use this to run tracie in a new job I'm working on. Reviewed-by: Christian Gmeiner <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5393>
* ci: fix possible spuriously run of jobsChristian Gmeiner2020-06-101-0/+2
| | | | | | | | | | | | Need to list arm_test-base here as well, or jobs using this template may spuriously run if the arm_test-base job fails or is cancelled. Suggested-by: Michel Dänzer <[email protected]> Signed-off-by: Christian Gmeiner <[email protected]> Reviewed-by: Michel Dänzer <[email protected]> Reviewed-by: Daniel Stone <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5405>
* ci: Use rsync for initial nfsroot population on cheza.Eric Anholt2020-06-091-2/+2
| | | | | | | | | | rm -rf and then copying over all the contents again is a waste of time when we'll almost always be using the same rootfs. Saves about 30s of job time. Closes: #3065 Reviewed-by: Christian Gmeiner <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5266>
* ci: Enable pre-merge fractional vulkan CTS runs on the turnip driver.Eric Anholt2020-06-091-0/+26
| | | | | | | | | Test 1/50th of the CTS on a630 pre-merge, since we've got hardware that can do it and infrastructure that should handle instability with a less-mature driver. Reviewed-by: Christian Gmeiner <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5266>
* ci: Build the full VK CTS for baremetal testing.Eric Anholt2020-06-091-2/+2
| | | | | | | | | | | | I'm going to enable the VK CTS on cheza, so swap the deqp we have in the container. build-deqp-vk already included GLES deqp binaries and data, and is a newer branch than the last opengl-es-cts tag. This brings a few things back over from build-deqp-gl for testlog extraction, and copyes out the GLES mustpass lists. Reviewed-by: Christian Gmeiner <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5266>
* ci: Bump up to the current version of the VK CTS.Eric Anholt2020-06-091-1/+1
| | | | | | | | For enabling VK CTS on freedreno, I've heard there were important stability fixes in the CTS recently. Reviewed-by: Samuel Pitoiset <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5266>
* ci: use separate docker images for baremetal buildsChristian Gmeiner2020-06-091-10/+10
| | | | | | | | | Using arm_test-base as a separate base layer as well for storage & network bandwidth efficiency. Signed-off-by: Christian Gmeiner <[email protected]> Reviewed-by: Eric Anholt <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5381>
* ci: add arm_test-base docker imageChristian Gmeiner2020-06-091-2/+22
| | | | | | | | Similar to x86_build-base. Signed-off-by: Christian Gmeiner <[email protected]> Reviewed-by: Eric Anholt <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5381>
* ci: bump virglrenderer to latest versionDave Airlie2020-06-041-1/+1
| | | | | | | Need this for upcoming GL 4.0 llvmpipe support. Reviewed-by: Elie Tournier <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5323>
* llvmpipe: move coroutines out of noopt caseDave Airlie2020-06-041-0/+1
| | | | | | | | | | | | | | | | | the virgl CI code was using the noopt path and crashing with a wierd can't select llvm.coro.subfn.addr error, turns out we have to call the cleanup pass no matter what. This enable a lot more virgl gles31 passes, but we have to disable tessellation shaders as now they executed, they crash due to missing OES_gpu_shader5, I should try and reenable them when llvmpipe is further along Fixes: d32690b43c91d ("gallivm: add coroutine pass manager support") Reviewed-by: Roland Scheidegger <[email protected]> Acked-by: Elie Tournier <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5320>
* gitlab-ci: bump piglit checkout commitTimothy Arceri2020-06-031-1/+1
| | | | Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4797>
* nir: Fold f2f16(b2f32(x)) to b2f16(x)Alyssa Rosenzweig2020-06-021-1/+0
| | | | | | | | | | | | | By definition. This reduces register pressure on freedreno so that the noubo expected failure goes away. Signed-off-by: Alyssa Rosenzweig <[email protected]> Reviewed-by: Marek Olšák <[email protected]> Reviewed-by: Rob Clark <[email protected]> Reviewed-by: Eric Anholt <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5002>
* gitlab-ci: Use separate docker images for cross buildsMichel Dänzer2020-06-021-13/+52
| | | | | | | | | | | Using x86_build-base as a separate base layer as well for storage & network bandwidth efficiency. Using separate images allows dropping the workarounds from the cross build job scripts. Reviewed-by: Eric Anholt <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5296>
* gitlab-ci: Add x86_build-base docker imageMichel Dänzer2020-06-021-7/+28
| | | | | | | Similar to x86_test-base. Reviewed-by: Eric Anholt <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5296>
* gitlab-ci: Stop using packages from Debian testingMichel Dänzer2020-06-021-3/+3
| | | | | | | | Not needed anymore (for now?). Reviewed-by: Erik Faye-Lund <[email protected]> Reviewed-by: Eric Anholt <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5296>
* gitlab-ci: Use Debian 10 wine-development packagesMichel Dänzer2020-06-021-2/+2
| | | | | | | | They're version 4.2, new enough for the MinGW job tests to pass. Reviewed-by: Erik Faye-Lund <[email protected]> Reviewed-by: Eric Anholt <[email protected]> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5296>
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-01 20:23:00 +0000