Yesterday I finally deleted all Fedora-AtomicHost AMIs and associated snapshots (it took whole night to finish).
This time, I know we have to start with AMIs first (and only then delete snapshots).
Where I can continue witht the cleanup? There is several dozen thousand of AMIs. At the end of this email I will give random sample from the list.
I am very afraid of deleting something that is still currently in use and that is somewhere listed as golden image.
Or we do not care about anything but images of stable Fedoras and everything that matches 'Fedora.*-X-.*' where X is number bellow < 38?
Miroslav
* ami-26c3b846 Fedora-Cloud-Base-24-20160601.n.0.x86_64-us-west-1-HVM-standard-0 * ami-05fd9470f34e60fdc Fedora-Cloud-Base-39-1.1.aarch64-hvm-us-west-1-gp3-0 * ami-062b2a2f58fdaf72f fedora-coreos-36.20220820.2.0-aarch64 * ami-f4bbff94 Fedora-Cloud-Atomic-23-20160626.x86_64-us-west-1-HVM-standard-0 * ami-31da9d51 Fedora-Atomic-24-20160706.0.x86_64-us-west-1-HVM-gp2-0 * ami-0bd20e7557d7bdbb6 Fedora-Cloud-Base-29-20190726.0.aarch64-hvm-us-west-1-standard-0 * ami-20a3a640 Fedora-Atomic-27-20171211.0.x86_64-us-west-1-HVM-gp2-0 * ami-fab2c59a Fedora-Cloud-Base-23-20160127.2.x86_64-us-west-1-PV-standard-0 * ami-981812f8 Fedora-Cloud-Base-27-20180303.0.x86_64-us-west-1-HVM-gp2-0 * ami-2134cb65 Fedora-Cloud-Base-23_Alpha-20150806.2.x86_64-us-west-1-HVM-gp2-0 * ami-0b8037b9dcf49e74e fedora-coreos-39.20231101.1.0-x86_64 * ami-312c0f51 Fedora-Atomic-25-20170601.0.x86_64-us-west-1-HVM-gp2-0 * ami-bacaccda Fedora-Atomic-26-20171226.0.x86_64-us-west-1-HVM-gp2-0 * ami-0b363a6b Fedora-Cloud-Base-26-20180129.0.x86_64-us-west-1-HVM-gp2-0 * ami-b2909cd2 Fedora-Atomic-Rawhide-20180131.n.0.x86_64-us-west-1-HVM-standard-0 * ami-04165988107e37bff Fedora-Cloud-Base-28-20190510.0.x86_64-hvm-us-west-1-standard-0 * ami-0b4577fc26dfa0a41 fedora-coreos-38.20230722.1.0-aarch64 * ami-0f46b6ee06ec9518a fedora-coreos-36.20220505.2.0-x86_64 * ami-3a30315a Fedora-Cloud-Base-26-20180101.0.x86_64-us-west-1-HVM-standard-0 * ami-089d45c0dde166fe0 Fedora-Cloud-Base-30-20190728.0.aarch64-hvm-us-west-1-gp2-0 * ami-eb84ba8b Fedora-Atomic-Rawhide-20171112.n.0.x86_64-us-west-1-HVM-standard-0 * ami-05254bd86175b9023 fedora-coreos-35.20220424.3.0-x86_64 * ami-f5ecdc95 Fedora-Atomic-26-20171001.0.x86_64-us-west-1-HVM-gp2-0 * ami-c2ab83a2 Fedora-Cloud-Base-25-20170802.0.x86_64-us-west-1-HVM-standard-0 * ami-df3367bf Fedora-Atomic-25-20161121.0.x86_64-us-west-1-HVM-standard-0 * ami-0419c7c8be07bf733 Fedora-Cloud-Base-30-20190419.n.0.x86_64-hvm-us-west-1-standard-0 * ami-c20612a2 Fedora-Atomic-27-20180314.0.x86_64-us-west-1-HVM-standard-0 * ami-a4192ec4 Fedora-Atomic-26-20170908.0.x86_64-us-west-1-HVM-gp2-0 * ami-9dc1f1fd Fedora-Cloud-Base-27-20171001.n.2.x86_64-us-west-1-HVM-gp2-0 * ami-d90255b9 Fedora-Cloud-Base-25-20161130.1.x86_64-us-west-1-PV-gp2-0 * ami-72747712 Fedora-Cloud-Base-Rawhide-20180110.n.0.x86_64-us-west-1-HVM-gp2-0 * ami-885052e8 Fedora-Cloud-Base-Rawhide-20180115.n.0.x86_64-us-west-1-HVM-gp2-0 * ami-5f5c643f Fedora-Cloud-Base-Rawhide-20171118.n.1.x86_64-us-west-1-HVM-gp2-0 * ami-4b115a2b Fedora-Cloud-Base-24-20161021.0.x86_64-us-west-1-PV-standard-0 * ami-fa40379a Fedora-Cloud-Base-23-20160127.1.x86_64-us-west-1-HVM-standard-0 * ami-04aacbcff5e34df3b Fedora-Cloud-Base-32-20200605.0.x86_64-hvm-us-west-1-gp2-0 * ami-04f2a5462112fc48f fedora-coreos-36.20220820.3.0-x86_64 * ami-9c5a20fc Fedora-Cloud-Base-23-20160605.x86_64-us-west-1-HVM-gp2-0 * ami-50c5f530 Fedora-Cloud-Base-25-20171002.0.x86_64-us-west-1-PV-standard-0 * ami-db6f6fbb Fedora-Atomic-Rawhide-20180105.n.0.x86_64-us-west-1-HVM-gp2-0 * ami-06ce8f3b14afca272 Fedora-Cloud-Base-30-20190323.n.0.aarch64-hvm-us-west-1-standard-0 * ami-0fc1ba6f Fedora-Atomic-24-20160601.n.0.x86_64-us-west-1-HVM-gp2-0 * ami-6c54550c Fedora-Atomic-Rawhide-20171228.n.0.x86_64-us-west-1-HVM-standard-0 * ami-a2591ac2 Fedora-Cloud-Base-24-20160809.0.x86_64-us-west-1-PV-standard-0 * ami-b31743d3 Fedora-Cloud-Base-25-20161119.3.x86_64-us-west-1-PV-standard-0 * ami-df4f6cbf Fedora-Atomic-26_Beta-1.3.x86_64-us-west-1-HVM-standard-0 * ami-e790d387 Fedora-Cloud-Base-24-20160811.0.x86_64-us-west-1-PV-standard-0 * ami-b3ddc9d3 Fedora-Cloud-Base-27-20180315.0.x86_64-us-west-1-HVM-gp2-0 * ami-273c7947 Fedora-Cloud-Base-23-20160613.x86_64-us-west-1-PV-gp2-0 * ami-594c1b39 Fedora-Cloud-Base-24-20161203.0.x86_64-us-west-1-PV-standard-0 * ami-203a3e40 Fedora-Cloud-Base-27-20171214.0.x86_64-us-west-1-HVM-gp2-0 * ami-b3b388d3 Fedora-Atomic-27-20171129.0.x86_64-us-west-1-HVM-standard-0 * ami-d1e1f0b1 Fedora-Cloud-Base-28-20180331.n.1.x86_64-us-west-1-HVM-gp2-0 * ami-063bd28aeead1fb4e Fedora-Cloud-Base-30-20190509.0.x86_64-hvm-us-west-1-gp2-0 * ami-43464723 Fedora-Atomic-26-20171229.0.x86_64-us-west-1-HVM-standard-0 * ami-6b38160b Fedora-Cloud-Base-24-20170713.0.x86_64-us-west-1-PV-gp2-0 * ami-012a0519ada69f3f9 Fedora-Cloud-Base-39-1.1.x86_64-hvm-us-west-1-gp3-0 * ami-04ee6790c93aa9851 Fedora-Cloud-Base-Rawhide-20190506.n.1.x86_64-hvm-us-west-1-gp2-0 * ami-0417e1efaaf6a4fa0 Fedora-Cloud-Base-29-20190703.0.x86_64-hvm-us-west-1-gp2-0 * ami-1e665f7e Fedora-Atomic-Rawhide-20171113.n.0.x86_64-us-west-1-HVM-gp2-0 * ami-5b5b1c3b Fedora-Cloud-Base-24-20160701.0.x86_64-us-west-1-PV-standard-0 * ami-0310f1a5d2854c18d fedora-coreos-37.20230218.2.0-x86_64 * ami-7cba961c Fedora-Cloud-Base-25-20170630.0.x86_64-us-west-1-HVM-gp2-0 * ami-b20204d2 Fedora-Cloud-Base-26-20171224.0.x86_64-us-west-1-HVM-gp2-0 * ami-01f5acf38212d009a fedora-coreos-38.20230430.1.0-aarch64 * ami-8f023fef Fedora-Atomic-26-20171021.0.x86_64-us-west-1-HVM-gp2-0 * ami-e28da282 Fedora-Cloud-Base-25-20170709.0.x86_64-us-west-1-PV-gp2-0 * ami-0dd31964c3588f1be fedora-coreos-37.20220918.1.1-aarch64 * ami-5ea6e13e Fedora-Atomic-24-20160704.0.x86_64-us-west-1-HVM-gp2-0 * ami-0ea8cdcd48ebca78c fedora-coreos-38.20230430.2.1-x86_64 * ami-3c10555c Fedora-Cloud-Base-24-1.1.x86_64-us-west-1-HVM-standard-0 * ami-7f71661f Fedora-Cloud-Base-26-20180318.0.x86_64-us-west-1-HVM-standard-0 * ami-c37b46a3 Fedora-Cloud-Base-25-20171018.0.x86_64-us-west-1-PV-gp2-0 * ami-bf7343df Fedora-Atomic-26-20170927.0.x86_64-us-west-1-HVM-gp2-0 * ami-dc9da0bc Fedora-Cloud-Base-26-20171024.0.x86_64-us-west-1-HVM-gp2-0 * ami-02fb010ee671c467d fedora-coreos-35.20220103.2.0-x86_64 * ami-38291258 Fedora-Cloud-Base-Rawhide-20171126.n.0.x86_64-us-west-1-HVM-standard-0 * ami-847f30e4 Fedora-Cloud-Base-24-20160916.0.x86_64-us-west-1-HVM-standard-0 * ami-dc180fbc Fedora-Cloud-Base-26-20180319.0.x86_64-us-west-1-HVM-standard-0 * ami-07b653d82033f9681 fedora-coreos-39.20231002.1.1-x86_64 * ami-4c64312c Fedora-Cloud-Base-24-20161114.0.x86_64-us-west-1-PV-gp2-0 * ami-0920ba29d7e90437f Fedora-Cloud-Base-31-20191112.0.aarch64-hvm-us-west-1-standard-0 * ami-0bb6684d9f09744e3 Fedora-Cloud-Base-39-20240206.0.aarch64-hvm-us-west-1-gp3-0 * ami-c46767a4 Fedora-Cloud-Base-27-20180105.3.x86_64-us-west-1-HVM-standard-0 * ami-050ea365519bc0db0 Fedora-Cloud-Base-30-20190322.n.0.aarch64-hvm-us-west-1-standard-0 * ami-088cebf2442fc4894 fedora-coreos-38.20230609.1.0-aarch64 * ami-3993d059 Fedora-Atomic-24-20160811.0.x86_64-us-west-1-HVM-gp2-0 * ami-59226639 Fedora-Cloud-Base-23-20160621.x86_64-us-west-1-PV-standard-0 * ami-f8246298 Fedora-Atomic-24-20160708.0.x86_64-us-west-1-HVM-standard-0 * ami-e4c9b384 Fedora-Atomic-24-20160610.n.0.x86_64-us-west-1-HVM-standard-0 * ami-05a7604cc4aabbd2a Fedora-Cloud-Base-30-20190406.n.1.x86_64-hvm-us-west-1-gp2-0 * ami-0d72747c5ac532500 Fedora-Cloud-Base-Rawhide-20240205.n.0.x86_64-hvm-us-west-1-gp3-0 * ami-f6fbfb96 Fedora-Cloud-Base-Rawhide-20180108.n.0.x86_64-us-west-1-HVM-standard-0 * ami-4c152b2c Fedora-Cloud-Base-26-20171109.0.x86_64-us-west-1-HVM-gp2-0 * ami-066b5c323aff4ece7 Fedora-Cloud-Base-30-20190515.0.x86_64-hvm-us-west-1-gp2-0 * ami-0ce1ef8681128ac17 CentOS Stream 8 aarch64 20240101 * ami-68ede108 Fedora-Cloud-Base-27-20180130.0.x86_64-us-west-1-HVM-gp2-0 * ami-71f0e311 Fedora-Cloud-Base-28-20180410.n.1.x86_64-paravirtual-us-west-1-standard-0 * ami-0d23bcbbbc029d3cd fedora-coreos-34.20210418.1.0 * ami-c63c17a6 Fedora-Cloud-Base-24-20170807.0.x86_64-us-west-1-HVM-gp2-0 * ami-00fcca60 Fedora-Atomic-25-20170917.0.x86_64-us-west-1-HVM-standard-0 * ami-07d79dbf8e2b414a3 Fedora-Cloud-Base-32-20200606.0.x86_64-hvm-us-west-1-standard-0 * ami-094c894d Fedora-Cloud-Base-23_Beta-20150915.x86_64-us-west-1-HVM-standard-1 * ami-428db622 Fedora-Cloud-Base-27-20171130.0.x86_64-us-west-1-HVM-gp2-0 * ami-d797b8b7 Fedora-Cloud-Base-25-20170710.0.x86_64-us-west-1-PV-gp2-0 * ami-04426964 Fedora-Cloud-Base-24-20170805.0.x86_64-us-west-1-HVM-standard-0
Dne 08. 02. 24 v 17:09 Miroslav Suchý napsal(a):
Where I can continue witht the cleanup? There is several dozen thousand of AMIs. At the end of this email I will give random sample from the list.
For the record, here is complete list containing 150k lines [9MB]
Thanks for taking that on Miroslav!
On Thu, 8 Feb 2024 at 16:24, Miroslav Suchý msuchy@redhat.com wrote:
Dne 08. 02. 24 v 17:09 Miroslav Suchý napsal(a):
Where I can continue witht the cleanup? There is several dozen thousand
of AMIs. At the end of this email I will give
random sample from the list.
For the record, here is complete list containing 150k lines [9MB]
--
Miroslav Suchy, RHCA Red Hat, Manager, Packit and CPT, #brno, #fedora-buildsys -- _______________________________________________ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedorapro... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Thu, Feb 08, 2024 at 05:09:27PM +0100, Miroslav Suchý wrote:
Yesterday I finally deleted all Fedora-AtomicHost AMIs and associated snapshots (it took whole night to finish).
This time, I know we have to start with AMIs first (and only then delete snapshots).
Where I can continue witht the cleanup? There is several dozen thousand of AMIs. At the end of this email I will give random sample from the list.
I am very afraid of deleting something that is still currently in use and that is somewhere listed as golden image.
Or we do not care about anything but images of stable Fedoras and everything that matches 'Fedora.*-X-.*' where X is number bellow < 38?
So... my take:
I think we should leave "GA" images. Even thought they are EOL for the most part, I think it's still possibly nice to be able to spin one up to test something or the like. We can find the names on our download server, ie,
https://dl.fedoraproject.org/pub/archive/fedora/linux/releases/35/Cloud/x86_... Fedora-Cloud-Base-35-1.2 is the GA for fedora 35 cloud.
We should exclude all 'current' releases (ie, 38/39/40)
We should exclude "Rawhide" ones that are 2024? I don't think we need to keep all the old ones there. We have them koji if we really need them. (At least the last month or two)
Perhaps for the coreos ones use similar rules? Dusty?
I am unsure about the CentOS ones. We should check with them on that.
Would it be worth it to rename the ones we plan to delete with a 'about to delete' name, wait a while and then delete? Or is there any way to tell who/how many people are using a ami?
kevin
Dne 09. 02. 24 v 18:27 Kevin Fenzi napsal(a):
I think we should leave "GA" images. Even thought they are EOL for the most part, I think it's still possibly nice to be able to spin one up to test something or the like. We can find the names on our download server, ie,
https://dl.fedoraproject.org/pub/archive/fedora/linux/releases/35/Cloud/x86_... Fedora-Cloud-Base-35-1.2 is the GA for fedora 35 cloud.
Nod. I was about to ask how can I find them... but the name match nicely. And going manualy over 35 names is likely not big deal.
I will tag them. Then they disappear from my radar.
I propose tag
FedoraGroup=ga-archives
Any objections?
We should exclude all 'current' releases (ie, 38/39/40)
*nod*
We should exclude "Rawhide" ones that are 2024? I don't think we need to keep all the old ones there. We have them koji if we really need them. (At least the last month or two)
*nod*
I am unsure about the CentOS ones. We should check with them on that.
I want to put CentOS aside for now. There is "only" 2k out of 145k that are related to centos. We can work on them in a later step.
Would it be worth it to rename the ones we plan to delete with a 'about to delete' name, wait a while and then delete? Or is there any way to tell who/how many people are using a ami?
All operation are reference to AMI are using ami-id. If I change name, likely no one will a notice.
But I can do what I did with volumes - first tag it with FedoraGroup=garbage-collector and only then delete it. This can lower the human error on my side.
I can rename too. That will be no problem for me, but I would not bet on that somebody notice of diferent name.
Dne 09. 02. 24 v 20:34 Miroslav Suchý napsal(a):
I think we should leave "GA" images. Even thought they are EOL for the most part, I think it's still possibly nice to be able to spin one up to test something or the like. We can find the names on our download server, ie,
https://dl.fedoraproject.org/pub/archive/fedora/linux/releases/35/Cloud/x86_... Fedora-Cloud-Base-35-1.2 is the GA for fedora 35 cloud.
Nod. I was about to ask how can I find them... but the name match nicely. And going manualy over 35 names is likely not big deal.
I will tag them. Then they disappear from my radar.
I propose tag
FedoraGroup=ga-archives
Any objections?
I tagged all GA images with this ^^^ tag.
I went from Fedora 39 down to Fedora 19. But I did not find any image for Fedora 19 and 20 (that is year 2013) so I stopped there.
I label AMIs and associated snapshots.
For the record, this is the the script I used for labeling the AMI in all regions https://github.com/xsuchy/fedora-infra-scripts/blob/main/label-ami.py
Who is responsible for uploading Fedora Cloud images to AWS? Fedora Cloud SIG? Somebody else? I want to make sure that consequent GA images, will be properly tagged.
On Mon, Feb 12, 2024 at 08:40:32PM +0100, Miroslav Suchý wrote:
Dne 09. 02. 24 v 20:34 Miroslav Suchý napsal(a):
I think we should leave "GA" images. Even thought they are EOL for the most part, I think it's still possibly nice to be able to spin one up to test something or the like. We can find the names on our download server, ie,
https://dl.fedoraproject.org/pub/archive/fedora/linux/releases/35/Cloud/x86_... Fedora-Cloud-Base-35-1.2 is the GA for fedora 35 cloud.
Nod. I was about to ask how can I find them... but the name match nicely. And going manualy over 35 names is likely not big deal.
I will tag them. Then they disappear from my radar.
I propose tag
FedoraGroup=ga-archives
Any objections?
I tagged all GA images with this ^^^ tag.
Thanks.
I went from Fedora 39 down to Fedora 19. But I did not find any image for Fedora 19 and 20 (that is year 2013) so I stopped there.
I label AMIs and associated snapshots.
For the record, this is the the script I used for labeling the AMI in all regions https://github.com/xsuchy/fedora-infra-scripts/blob/main/label-ami.py
Who is responsible for uploading Fedora Cloud images to AWS? Fedora Cloud SIG? Somebody else? I want to make sure that consequent GA images, will be properly tagged.
Well, it's done with the fedimg app in fedora infra currently, so thats where it would need to change right now.
The cloud sig has plans to replace fedimg, which I hope they can do well before rhel7 eol in a few months, because fedimg is python2/rhel7.
kevin
infrastructure@lists.fedoraproject.org
-
David Kirwan -
Kevin Fenzi -
Miroslav Suchý