Didn't see this go by, but it looks hot enough to risk a repeat posting. From a friend:
It appears there's been a very serious effort to backdoor sshd on Linux via the xz compression/decompression system.
https://www.openwall.com/lists/oss-security/2024/03/29/4
If you have anything running very recent Linux, it's worth investigating whether you're affected.
IBM Red Hat says, if you're running Fedora 40 or Fedora Rawhide:
PLEASE IMMEDIATELY STOP USAGE OF ANY FEDORA RAWHIDE INSTANCES for work or personal activity.
The identity that did this got to the point of being not only an xz maintainer but a Linux kernel contributor, and contributed to a number of other Open Source projects as well over the course of years. The identity may have been compromised to do this, or may have been created to do this, and may have used other contributions to build rapport or to compromise more projects as well.
I looked at the detection script available at the URL in the posting. It's harmless at worst (don't know yet if it can detect anything).
Caveat Utilitor, -- Dave Ihnat
On Sat, 2024-03-30 at 12:08 -0500, Dave Ihnat wrote:
Didn't see this go by, but it looks hot enough to risk a repeat posting. From a friend:
It appears there's been a very serious effort to backdoor sshd on Linux via the xz compression/decompression system.
https://www.openwall.com/lists/oss-security/2024/03/29/4
If you have anything running very recent Linux, it's worth investigating whether you're affected.
AFAIK this only applies to Rawhide and the (as yet unreleased) F40, both of which I assume will be patched ASAP.
poc
On 30 Mar 2024, at 17:16, Patrick O'Callaghan pocallaghan@gmail.com wrote:
AFAIK this only applies to Rawhide and the (as yet unreleased) F40, both of which I assume will be patched ASAP.
F40 beta already the reverted to the older version of xz. I was able to update my beta f40 earlier today.
Barry
On Mar 30, 2024, at 13:16, Patrick O'Callaghan pocallaghan@gmail.com wrote:
On Sat, 2024-03-30 at 12:08 -0500, Dave Ihnat wrote:
Didn't see this go by, but it looks hot enough to risk a repeat posting. From a friend:
It appears there's been a very serious effort to backdoor sshd on Linux via the xz compression/decompression system.
https://www.openwall.com/lists/oss-security/2024/03/29/4
If you have anything running very recent Linux, it's worth investigating whether you're affected.
AFAIK this only applies to Rawhide and the (as yet unreleased) F40, both of which I assume will be patched ASAP.
Thankfully, it looks like the version that was released in the Fedora 40 beta repos (v5.6.0) was compiled with a configure flag that prevented the backdoor from running, because the malicious code unintentionally caused Fedora’s QA process to reject the initial updated package (if I understand correctly). Upstream released a new version that allowed Fedora to build with the feature, it just didn’t make it in the beta freeze. Complete coincidence. Fedora has since reverted the xz packages to v5.4.6 in 40, so if you’re running the beta, you can `dnf downgrade xz*’ to get the older version, if it doesn’t automatically downgrade.
We are pretty sure there are no other backdoors in xz or liblzma, but all the contributions by this author are getting heavy scrutiny. Some distros are even discussing reverting xz back until the version before the malicious co-maintainer joined the project, which will require significant effort.
Major props to the Fedora team for handling this, and the security team at Red Hat who were involved with the discovery and investigation. We should also all thank Andres Freund for his meticulous discovery of the backdoor, without which, we might have ended up with it he backdoor running in production for many distros.
On Sat, Mar 30, 2024 at 3:01 PM Jonathan Billings billings@negate.org wrote:
On Mar 30, 2024, at 13:16, Patrick O'Callaghan pocallaghan@gmail.com wrote:
On Sat, 2024-03-30 at 12:08 -0500, Dave Ihnat wrote:
Didn't see this go by, but it looks hot enough to risk a repeat posting. From a friend:
It appears there's been a very serious effort to backdoor sshd on Linux via the xz compression/decompression system.
https://www.openwall.com/lists/oss-security/2024/03/29/4
If you have anything running very recent Linux, it's worth investigating whether you're affected.
AFAIK this only applies to Rawhide and the (as yet unreleased) F40, both of which I assume will be patched ASAP.
Thankfully, it looks like the version that was released in the Fedora 40 beta repos (v5.6.0) was compiled with a configure flag that prevented the backdoor from running, because the malicious code unintentionally caused Fedora’s QA process to reject the initial updated package (if I understand correctly). Upstream released a new version that allowed Fedora to build with the feature, it just didn’t make it in the beta freeze. Complete coincidence. Fedora has since reverted the xz packages to v5.4.6 in 40, so if you’re running the beta, you can `dnf downgrade xz*’ to get the older version, if it doesn’t automatically downgrade.
The last untainted version of xz is circa 5.2. Starting around version 5.4, Jia Tan was making commits. And version 5.3 was a developer/debug build, so you have to rewind a bit further to 5.2. Also see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1068024#5.
The next problem free release with ABI and symbol compat should be version 5.6.2 or above. I would tag it 5.7 or 6.0 since it is a major milestone (with the mark being backdoor-free code). There's no telling when Lasse releases that, however.
We are pretty sure there are no other backdoors in xz or liblzma, but all the contributions by this author are getting heavy scrutiny. Some distros are even discussing reverting xz back until the version before the malicious co-maintainer joined the project, which will require significant effort.
Major props to the Fedora team for handling this, and the security team at Red Hat who were involved with the discovery and investigation. We should also all thank Andres Freund for his meticulous discovery of the backdoor, without which, we might have ended up with it he backdoor running in production for many distros.
Yeah, nice investigative work.
Jeff
On 3/30/24 12:00, Jonathan Billings wrote:
On Mar 30, 2024, at 13:16, Patrick O'Callaghan pocallaghan@gmail.com wrote: AFAIK this only applies to Rawhide and the (as yet unreleased) F40, both of which I assume will be patched ASAP.
Thankfully, it looks like the version that was released in the Fedora 40 beta repos (v5.6.0) was compiled with a configure flag that prevented the backdoor from running, because the malicious code unintentionally caused Fedora’s QA process to reject the initial updated package (if I understand correctly). Upstream released a new version that allowed Fedora to build with the feature, it just didn’t make it in the beta freeze. Complete coincidence. Fedora has since reverted the xz packages to v5.4.6 in 40, so if you’re running the beta, you can `dnf downgrade xz*’ to get the older version, if it doesn’t automatically downgrade.
The epoch was bumped, so an upgrade will get the "older" version. Don't try to downgrade.
On Sat, Mar 30, 2024 at 1:08 PM Dave Ihnat dihnat@dminet.com wrote:
Didn't see this go by, but it looks hot enough to risk a repeat posting. From a friend:
It appears there's been a very serious effort to backdoor sshd on Linux via the xz compression/decompression system.
https://www.openwall.com/lists/oss-security/2024/03/29/4
If you have anything running very recent Linux, it's worth investigating whether you're affected.
Lasse Collin, the author of xz, published a statement at https://tukaani.org/xz-backdoor/.
And to be clear, the bad actor is Jia Tan. Lasse appears to be collateral damage.
IBM Red Hat says, if you're running Fedora 40 or Fedora Rawhide:
PLEASE IMMEDIATELY STOP USAGE OF ANY FEDORA RAWHIDE INSTANCES for work or personal activity.
The identity that did this got to the point of being not only an xz maintainer but a Linux kernel contributor, and contributed to a number of other Open Source projects as well over the course of years. The identity may have been compromised to do this, or may have been created to do this, and may have used other contributions to build rapport or to compromise more projects as well.
Jia Tan pulled his shenanigans on libarchive, too: https://github.com/libarchive/libarchive/pull/1609.
I looked at the detection script available at the URL in the posting. It's harmless at worst (don't know yet if it can detect anything).
Here are Debian and Gentoo bugs tracking the issue:
* https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1068024 * https://bugs.gentoo.org/928134
Jeff
On Sat, Mar 30, 2024 at 1:08 PM Dave Ihnat dihnat@dminet.com wrote:
Didn't see this go by, but it looks hot enough to risk a repeat posting. From a friend:
It appears there's been a very serious effort to backdoor sshd on Linux via the xz compression/decompression system.
https://www.openwall.com/lists/oss-security/2024/03/29/4
If you have anything running very recent Linux, it's worth investigating whether you're affected.
IBM Red Hat says, if you're running Fedora 40 or Fedora Rawhide:
PLEASE IMMEDIATELY STOP USAGE OF ANY FEDORA RAWHIDE INSTANCES for work or personal activity.
The identity that did this got to the point of being not only an xz maintainer but a Linux kernel contributor, and contributed to a number of other Open Source projects as well over the course of years. The identity may have been compromised to do this, or may have been created to do this, and may have used other contributions to build rapport or to compromise more projects as well.
I looked at the detection script available at the URL in the posting. It's harmless at worst (don't know yet if it can detect anything).
It looks like more analysis has revealed this is a RCE with the payload in the modulus of a public key: "The payload is extracted from the N value (the public key) passed to RSA_public_decrypt, checked against a simple fingerprint, and decrypted with a fixed ChaCha20 key before the Ed448 signature verification..." Also see https://www.openwall.com/lists/oss-security/2024/03/30/36.
Jeff
Yeah...this looks like a "big" issue...wonder what the resolution is?....removal?...or just hunker down and wait for a patch/update from the devs?...
https://youtu.be/tVvbLS2Bm8c?si=39dTmn4JD3YqYitU
On Sat, Mar 30, 2024, 4:08 PM Jeffrey Walton noloader@gmail.com wrote:
On Sat, Mar 30, 2024 at 1:08 PM Dave Ihnat dihnat@dminet.com wrote:
Didn't see this go by, but it looks hot enough to risk a repeat posting. From a friend:
It appears there's been a very serious effort to backdoor sshd on Linux via the xz compression/decompression system.
https://www.openwall.com/lists/oss-security/2024/03/29/4
If you have anything running very recent Linux, it's worth
investigating
whether you're affected.
IBM Red Hat says, if you're running Fedora 40 or Fedora Rawhide:
PLEASE IMMEDIATELY STOP USAGE OF ANY FEDORA RAWHIDE INSTANCES for work or personal activity.
The identity that did this got to the point of being not only an xz maintainer but a Linux kernel contributor, and contributed to a number of other Open Source projects as well over the course of years. The identity may have been compromised to do this, or may have been created to do this, and may have used other contributions to build rapport or to compromise more projects as well.
I looked at the detection script available at the URL in the posting.
It's
harmless at worst (don't know yet if it can detect anything).
It looks like more analysis has revealed this is a RCE with the payload in the modulus of a public key: "The payload is extracted from the N value (the public key) passed to RSA_public_decrypt, checked against a simple fingerprint, and decrypted with a fixed ChaCha20 key before the Ed448 signature verification..." Also see https://www.openwall.com/lists/oss-security/2024/03/30/36.
Jeff
users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On 3/30/24 14:31, Eddie O'Connor wrote:
Yeah...this looks like a "big" issue...wonder what the resolution is?....removal?...or just hunker down and wait for a patch/update from the devs?...
Updates are already available for the affected versions (rawhide and possibly F40 beta). Make sure you're updated and it's all fine.
On Sat, Mar 30, 2024 at 6:32 PM Eddie O'Connor eoconnor25@gmail.com wrote:
Yeah...this looks like a "big" issue...wonder what the resolution is?....removal?...or
just hunker down and wait for a patch/update from the devs?...
If you are one of few who installed the "bad" version, you don't have to wait, updates that replace the "bad" version have been released.
It could have been a big issue, but a just in time "accidental" discovery means few systems were affected, a detection script is available, and bad packages have been removed from repositories and updating will remove install "bad" packages. A few people may need to "clean" affected systems and regenerate keys.
This episode does, however, highlight underlying weaknesses of the open source ecosystem. Many open source projects are widely used but rely on unpaid developers. Some of the original developers are getting old or have other demands on their time. It appears to have been easy (perhaps too easy) for a well-funded and resourced entity to assume the role of an opensource developer.
I'm glad that there is a remedy and resolution, I will be checking "Mom's" Linux Mint laptop and my Fedora workstation laptop and desktop tonight..and if need be will perform triage procedures on all machines, I've been using Linux?...Fedora specifically....since 2003/04...and I've "survived" Spectre and even Dependency Hell, so I'm used to "big issues" coming up. Glad this isn't one of them, and while I'm not a developer?...I would LOVE to BE one!...as my son is now college bound and I don't have "babies" to tend to...I work from home...and if I could learn the framework and languages?..I would SO volunteer, I'm a "spry" 52 yr old...who's been in IT since '99... I guess we all have fantasy jobs though eh?
Thanks to all the devs and code maintainers who make Fedora a possibility for a dweeb lile me!! You guys and gals ROCK!!
On Sat, Mar 30, 2024, 6:29 PM George N. White III gnwiii@gmail.com wrote:
On Sat, Mar 30, 2024 at 6:32 PM Eddie O'Connor eoconnor25@gmail.com wrote:
Yeah...this looks like a "big" issue...wonder what the resolution is?....removal?...or
just hunker down and wait for a patch/update from the devs?...
If you are one of few who installed the "bad" version, you don't have to wait, updates that replace the "bad" version have been released.
It could have been a big issue, but a just in time "accidental" discovery means few systems were affected, a detection script is available, and bad packages have been removed from repositories and updating will remove install "bad" packages. A few people may need to "clean" affected systems and regenerate keys.
This episode does, however, highlight underlying weaknesses of the open source ecosystem. Many open source projects are widely used but rely on unpaid developers. Some of the original developers are getting old or have other demands on their time. It appears to have been easy (perhaps too easy) for a well-funded and resourced entity to assume the role of an opensource developer.
-- George N. White III
-- _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On 30 Mar at 17:46, Eddie O'Connor eoconnor25@gmail.com wrote:
...and while I'm not a developer?...I would LOVE to BE one!...as my son is now college bound and I don't have "babies" to tend to...I work from home...and if I could learn the framework and languages?..I would SO volunteer, I'm a "spry" 52 yr old...who's been in IT since '99...
I've been a developer since I got out of college ~1976. I don't know how spry I am, but I am 70 and still rockin' as my own consultant.
Actually--I was a full-time developer through around 2004, when I went out on my own. Incorporated my own business as an IT Consultant. Specialized in SMBs (Small/Medium Businesses), since I'd observed they get screwed by the consulting firms.
Since that time, I've done much less software development. Why? How many times can I rewrite the same solution, in different languages, for the same problems? That got tiring. I'm not saying that you shouldn't go for it--you *haven't* gone through my decades of development, and it's amazingly rewarding when you get in the groove.
I guess we all have fantasy jobs though eh?
Don't just treat it as fantasy. When I went to create my own company at 51, I had a friend who griped, "You can't do that! You're too old!". Foo on him. Go for what you want!
Thanks to all the devs and code maintainers who make Fedora a possibility for a dweeb lile me!! You guys and gals ROCK!!
I re-wrote "cut" and "paste" and submitted them to Gnu back in the '80s. It was both gratifying and amazingly painful (BTL lawyers were not best pleased. Fortunately, I did it "by the book"--got permission from my BTL consultant manager, made sure I didn't look at the original source code, etc.) so I ended up clean. Open Source is the way to keep things moving and surviving. If you want to get into it, DO IT!
Sincerely, -- Dave Ihnat
Dave, Eddie,
Good to hear your stories! - see inline comments:
On 2024-03-31 10:42, Dave Ihnat wrote:
On 30 Mar at 17:46, Eddie O'Connor eoconnor25@gmail.com wrote:
...and while I'm not a developer?...I would LOVE to BE one!...as my son is now college bound and I don't have "babies" to tend to...I work from home...and if I could learn the framework and languages?..I would SO volunteer, I'm a "spry" 52 yr old...who's been in IT since '99...
I've been a developer since I got out of college ~1976. I don't know how spry I am, but I am 70 and still rockin' as my own consultant.
Beat you both! - now 72 and started using the original RH4 but moved to F01 straight away - but I have actually been using Linux since the Kernel 0.9 days . .
Actually--I was a full-time developer through around 2004, when I went out on my own. Incorporated my own business as an IT Consultant. Specialized in SMBs (Small/Medium Businesses), since I'd observed they get screwed by the consulting firms.
Good work!
Since that time, I've done much less software development. Why? How many times can I rewrite the same solution, in different languages, for the same problems? That got tiring. I'm not saying that you shouldn't go for it--you *haven't* gone through my decades of development, and it's amazingly rewarding when you get in the groove.
Yes, I used to love building kernels etc too but after a while you do get short of time and resort to just "getting stuff done" as quickly as possible . .
I guess we all have fantasy jobs though eh?
Don't just treat it as fantasy. When I went to create my own company at 51, I had a friend who griped, "You can't do that! You're too old!". Foo on him. Go for what you want!
Exactly! +1
Thanks to all the devs and code maintainers who make Fedora a possibility for a dweeb lile me!! You guys and gals ROCK!!
From me too! - have loved this FOSS space for a long time!
I re-wrote "cut" and "paste" and submitted them to Gnu back in the '80s. It was both gratifying and amazingly painful (BTL lawyers were not best pleased. Fortunately, I did it "by the book"--got permission from my BTL consultant manager, made sure I didn't look at the original source code, etc.) so I ended up clean. Open Source is the way to keep things moving and surviving. If you want to get into it, DO IT!
Good work! - In the early days I helped debug the SCSI controller but that was about my limit of "serious" stuff . .
Kudos to all involved with this discovery and fix!
Regards,
Phil.
Sincerely,
Dave Ihnat
users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Thanks gentlemen....your my new inspiration!...I am making a conscious effort to make this my goal...and I will ONLY STOP?...
when I'm dead!!!
Carpe Diem!!!
On Sat, Mar 30, 2024, 7:42 PM Dave Ihnat dihnat@dminet.com wrote:
On 30 Mar at 17:46, Eddie O'Connor eoconnor25@gmail.com wrote:
...and while I'm not a developer?...I would LOVE to BE one!...as my son is now college bound and I don't have "babies" to tend to...I work from home...and if I could learn the framework and languages?..I would SO volunteer, I'm a "spry" 52 yr old...who's been in
IT
since '99...
I've been a developer since I got out of college ~1976. I don't know how spry I am, but I am 70 and still rockin' as my own consultant.
Actually--I was a full-time developer through around 2004, when I went out on my own. Incorporated my own business as an IT Consultant. Specialized in SMBs (Small/Medium Businesses), since I'd observed they get screwed by the consulting firms.
Since that time, I've done much less software development. Why? How many times can I rewrite the same solution, in different languages, for the same problems? That got tiring. I'm not saying that you shouldn't go for it--you *haven't* gone through my decades of development, and it's amazingly rewarding when you get in the groove.
I guess we all have fantasy jobs though eh?
Don't just treat it as fantasy. When I went to create my own company at 51, I had a friend who griped, "You can't do that! You're too old!". Foo on him. Go for what you want!
Thanks to all the devs and code maintainers who make Fedora a possibility for a dweeb lile me!! You guys and gals ROCK!!
I re-wrote "cut" and "paste" and submitted them to Gnu back in the '80s. It was both gratifying and amazingly painful (BTL lawyers were not best pleased. Fortunately, I did it "by the book"--got permission from my BTL consultant manager, made sure I didn't look at the original source code, etc.) so I ended up clean. Open Source is the way to keep things moving and surviving. If you want to get into it, DO IT!
Sincerely,
Dave Ihnat-- _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
-
Barry -
Dave Ihnat -
Eddie O'Connor -
George N. White III -
Jeffrey Walton -
Jonathan Billings -
Patrick O'Callaghan -
Philip Rhoades -
Samuel Sieb