Misc #17720: Cirrus CI to check non-x86_64 architecture cases by own machines

• Status: Assigned
• Assignee: jaruga (Jun Aruga)

[jaruga (Jun Aruga)]

Hello!

This ticket is related to the tickets #16234 #16360. But I opened a new ticket because it is related to general non-x86_64 architecture CI cases.

I have a suggestion.

I see the .travis.yml was removed [1], and I also saw another open source project remove their .travis.yml because they could not get the credits to continue to run Travis [2]. I feel Travis is not really a possible option for every open source project for now.

While we have RubyCI, I think we still have a motivation to run a CI on non-x86_64 architectures at a pull-request timing. So, I investigated alternative CI. When checking GitHub Actions, I do not feel it will happen soon on GitHub Actions [3]. Then I found an interesting CI called "Cirrus CI", that might enable us to run CI on non-x86_64 architectures such as Mac M1 (arm) ppc64le and s390x beyond the cloud.

Cirrus CI has 2 types of features: "cloud" and "persistent workers". I see the Cirrus CI "cloud" feature has been used in the QEMU and podman projects [4][5]. It has a unique freeBSD host. However the remarkable feature for the Ruby project is the "persistent workers" [6] announced a few months ago, that is beyond the cloud. Because this feature enables us to use our own machines as a CI running host. You can see the examples running the CI with the machines such as Mac M1, iPhone, ppc64le and s390x on the page [6]. Maybe the used machine does not even have the global static IP. You can see other articles [7][8] too.

I can see some benefits to start Cirrus CI for the Ruby project.

• Possibly we can check Mac M1 (arm), ppc64le, s390x cases using machines used in RubyCI [9] and someone's machine such as @ReiOdaira's ppc64le/s390x machines at the pull-request timing.
• When we face the CI issue, we can login to the machine and use the interactive debugging tool such as gdb to fix it.
• The config file is YAML format and it has the matrix feature [10]. We are familiar with the YAML and matrix.

What do you think? Positive or negative?
Thank you.

[1] ruby removed .travis.yml: https://github.com/ruby/ruby/commit/6b978d542704a5614af5e9375c4b31b8d2618652
[2] simde removed .travis.yml: https://github.com/simd-everywhere/simde/commit/17a27e7f2c3114225899f2ace14010cbbb2139b5
[3] GitHub Actions and ppc64le: https://github.community/t/self-hosted-runner-on-ppc64el-architecture/155337
[4] QEMU: https://gitlab.com/qemu-project/qemu/-/blob/master/.cirrus.yml
[5] Podman: https://github.com/containers/podman/blob/master/.cirrus.yml
[6] The issue ticket of Persistent Workers: https://github.com/cirruslabs/cirrus-ci-docs/issues/263#issuecomment-746900845
[7] Persistent Workers blog: https://medium.com/cirruslabs/announcing-public-beta-of-cirrus-ci-persistent-workers-7327a38004be
[8] Persistent Workers guide: https://cirrus-ci.org/guide/persistent-workers/
[9] RubyCI: https://rubyci.org/
[10] Cirrus CI matrix feature: https://cirrus-ci.org/guide/writing-tasks/#matrix-modification

[naruse (Yui NARUSE)]

While we have RubyCI, I think we still have a motivation to run a CI on non-x86_64 architectures at a pull-request timing.

Yes, we want non-x86_64 CIs with cheaper (maintenance/money) cost.
PR timing CI is desired rather than cron-based RubyCI.

When checking GitHub Actions, I do not feel it will happen soon on GitHub Actions [3].

Ah..., I didn't know GitHub Actions Self-hosted runner doesn't support ppc64...

Cirrus CI's persistent worker

It sounds interesting... I'll discuss with other core people.

[jaruga (Jun Aruga)]

While we have RubyCI, I think we still have a motivation to run a CI on non-x86_64 architectures at a pull-request timing.

Yes, we want non-x86_64 CIs with cheaper (maintenance/money) cost.
PR timing CI is desired rather than cron-based RubyCI.

Okay. For the cost of the maintenance, I remember on the ticket #16360 to add ppc64le/s390x, an issue that only happened on Travis ppc64le, but it was not reproduced on the local ppc64le environments prevented us from adding ppc64le to Travis. This kind of the cloud runner specific issue might be a cost in the cloud hosted runner, and might be improved in the self hosted runner.

When checking GitHub Actions, I do not feel it will happen soon on GitHub Actions [3].

Ah..., I didn't know GitHub Actions Self-hosted runner doesn't support ppc64...

I didn't know the GitHub Actions also has the self-hosted runner like Cirrus CI's persistent workers, and it supports the arm64 and arm32 (Linux only). Thanks for the info.
https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners

Cirrus CI's persistent worker

It sounds interesting... I'll discuss with other core people.

Thank you for the discussion!

[fkorotkov (Fedor Korotkov)]

Hello there,

I'm founder of Cirrus CI and will be happy to answer any of your questions and help you in any capacity.

Cirrus persistent workers are implemented via Cirrus CLI[1] which is written in Go so we can support any target that Go can compile to. If you'll find anything missing please also let me know so we can implement it for you.

[1] https://github.com/cirruslabs/cirrus-cli

[hsbt (Hiroshi SHIBATA)]

Interesting.

I and naruse (and others) hope to maintain non-x86 platform. But we don't have enough resources that are working time, cost and motivation.

@jaruga (Jun Aruga)

Do you interest to maintain it as a ruby committer?

@matz (Yukihiro Matsumoto)

Can we add jaruga as a committer? He will help to maintain Ruby language on the various platforms.

[mame (Yusuke Endoh)]

hsbt (Hiroshi SHIBATA) wrote in #note-4:

@jaruga (Jun Aruga)

Do you interest to maintain it as a ruby committer?

+1

[jaruga (Jun Aruga)]

Do you interest to maintain it as a ruby committer?

@hsbt (Hiroshi SHIBATA), @mame (Yusuke Endoh), yes! I am interested in maintaining Cirrus CI as a ruby committer.
My concern is that I am more comfortable to maintain the Linux use cases in the Cirrus CI. But I might need someone's help to maintain the MacOS and Windows use cases if those will be managed on the CI. Anyway, I can try to maintain the entire cases of the Cirrus CI for now.
If I do not notice the issue tickets related to the CI or the CI failures, please let me know. I will try to catch up.

[jaruga (Jun Aruga)]

I and naruse (and others) hope to maintain non-x86 platform. But we don't have enough resources that are working time, cost and motivation.

@hsbt (Hiroshi SHIBATA) did you mean I will maintain not only the Cirrus CI, but also the non-x86 platform generally? Yes, I am interested in it. :)

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Assigned
• Assignee set to matz (Yukihiro Matsumoto)

did you mean I will maintain not only the Cirrus CI

Yes, I means not only the Cirrus CI.

[jaruga (Jun Aruga)]

Yes, I means not only the Cirrus CI.

OK.

By the way, related to this topic, I found the following ticket asking the expanded platform support of GitHub Actions. The platforms are such as PowerPC, IBM Mainframe, Solaris, old Red Hat (5, 6) by rewriting the runner by Golang or Typescript. The answer from a staff is "for now we don't have plans to take on rewriting the runner".

Expanded platform support: Consider porting to go or typescript
https://github.com/actions/runner/issues/243#issuecomment-810672183

[naruse (Yui NARUSE)]

FYI
The current of the process is "Get the approval of matz about getting a commit right.".
See https://bugs.ruby-lang.org/projects/ruby/wiki/CommitterHowto for future processes.

My expectation is

• MUST: keep CIs green
• MUST: keep CIs are not slow. In this context "not slow" means "faster than other already exist CIs"
• Best effort: keep non-x86 platform stable
• Can: fix other bugs including non-x86 things. (For trivial fix, you can commit without discussion. For non-trivial, you should create ticket as usual, and after getting consensus you can commit it by yourself)

Note that keep CIs green is more important than stability of x86/non-x86 platform. To achieve that you can fix a commit which breaks something. If you cannot fix it, you can also just skip the test on certain platform or revert the commit with a URL of the CI failure in the revert commit message.

[matz (Yukihiro Matsumoto)]

I approve @jaruga (Jun Aruga) to be a member of the core committers.

Matz.

[jaruga (Jun Aruga)]

@naruse (Yui NARUSE) All right. I will take a look at the wiki page.

Thank you for the approval, Matz.

[jaruga (Jun Aruga)]

fkorotkov (Fedor Korotkov) wrote in #note-3:

Hello there,

I'm founder of Cirrus CI and will be happy to answer any of your questions and help you in any capacity.

Cirrus persistent workers are implemented via Cirrus CLI[1] which is written in Go so we can support any target that Go can compile to. If you'll find anything missing please also let me know so we can implement it for you.

[1] https://github.com/cirruslabs/cirrus-cli

Hi @fkorotkov (Fedor Korotkov), thanks for finding our Ruby project! Possibly we will let you know when we need your help.

[hsbt (Hiroshi SHIBATA)]

@jaruga (Jun Aruga)

Can you do the instructions of https://bugs.ruby-lang.org/projects/ruby/wiki/CommitterHowto ? After that, I will prepare your account on ruby-lang.org.

[jaruga (Jun Aruga)]

@hsbt (Hiroshi SHIBATA) yes, I sent the email by following the instructions now!

[jaruga (Jun Aruga)]

I just would update the current status.

Travis was revived with only free non-x86 cases. (#17818)

Here is a list of the possible servers that I plan to install Cirrus CI persistent workers.

• arm64 Ubuntu focal server, if we can use it sponsored by WorkOnArm project.
• ppc64le Ubuntu focal server sponsored by IBM.
• s390x Ubuntu focal server sponsored by IBM.
• M1 Mac (maybe later, after verifying the Cirrus CI on some of the above servers)

Right now we are discussing with the WorksOnArm project in a meeting and by emails. https://github.com/WorksOnArm/cluster/issues/262
Note the WorksOnArm project is organized by Arm and Equinix Metal (= formerly known as Packet).
We recently got free ppc64le and s390x Ubuntu focal servers sponsored by IBM.

[jeremyevans0 (Jeremy Evans)]

• Status changed from Assigned to Closed

Cirrus support added in dd78084ea6c3fe97302889eedcb04b859fdaf21c

[jaruga (Jun Aruga)]

• Status changed from Closed to Assigned
• Assignee changed from matz (Yukihiro Matsumoto) to jaruga (Jun Aruga)

Cirrus CI cloud hosting service was added to test Arm64 cases on the commit, because the Fedor@Cirrus CI contacted me, and it was a great offer. So, after sharing the info and seeing the reaction from other developers was positive on Slack jp channel, I added it. However this ticket's intent is originally to add self hosting type CI. So, please let me keep still open this ticket.