9:55 a.m.
Hello everyone,
I hope this email finds everyone well. We are currently packaging PyTorch for Fedora and
we are actively packaging the dependencies. We’re reaching out to see if anyone is
interested in joining the project. If you’re interested in more information, our meeting
notes are in a GitHub repo and we have the AI/ML SIG discussion page where we’ve hosted
all of our post meeting discussions. We have a WikiPage for our package assignment list if
you would like to see what needs packaging or just to see the status of the project. Our
next meeting is tomorrow, October 11th at 9AM EST and it’s on the FedoCal under the SIG
calendar. All lists are found below. If you’re interested in joining, have any questions,
or want to get a calendar invite for our reoccurring meeting, please feel free to email me
at kabdo(a)redhat.com or Teng at tema(a)redhat.com.
AI/ML SIG Discussion Page: https://discussion.fedoraproject.org/tag/ai-ml-sig
Meeting Notes: https://github.com/kaitlynabdo/pytorch-fedora-meeting-notes
Packaging Assignment List: https://fedoraproject.org/wiki/SIGs/PyTorch/packagingStatus
FedoCal: https://calendar.fedoraproject.org/SIGs/
Thank you!
Kaitlyn Abdo
11:41 p.m.
Congratulations for the PyTorch package!
https://src.fedoraproject.org/rpms/python-torch
I hope someone will announce this great achievement to the Fedora
community too, and update the following page too.
https://fedoraproject.org/wiki/SIGs/PyTorch/packagingStatus
Jun
On Wed, Oct 11, 2023 at 11:55 PM Kaitlyn Abdo <kabdo(a)redhat.com> wrote:
Hello everyone,
I hope this email finds everyone well. We are currently packaging PyTorch for Fedora and
we are actively packaging the dependencies. We’re reaching out to see if anyone is
interested in joining the project. If you’re interested in more information, our meeting
notes are in a GitHub repo and we have the AI/ML SIG discussion page where we’ve hosted
all of our post meeting discussions. We have a WikiPage for our package assignment list if
you would like to see what needs packaging or just to see the status of the project. Our
next meeting is tomorrow, October 11th at 9AM EST and it’s on the FedoCal under the SIG
calendar. All lists are found below. If you’re interested in joining, have any questions,
or want to get a calendar invite for our reoccurring meeting, please feel free to email me
at kabdo(a)redhat.com or Teng at tema(a)redhat.com.
AI/ML SIG Discussion Page: https://discussion.fedoraproject.org/tag/ai-ml-sig
Meeting Notes: https://github.com/kaitlynabdo/pytorch-fedora-meeting-notes
Packaging Assignment List: https://fedoraproject.org/wiki/SIGs/PyTorch/packagingStatus
FedoCal: https://calendar.fedoraproject.org/SIGs/
Thank you!
Kaitlyn Abdo
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
--
Jun | He - Him | Timezone: UTC+1 or 2, Czech Republic
See <https://www.worldtimebuddy.com/czech-republic-prague-to-utc> for
the timezone.
9:34 a.m.
On Friday, December 8, 2023 12:41:59 AM EST Jun Aruga (he / him) wrote:
Congratulations for the PyTorch package!
https://src.fedoraproject.org/rpms/python-torch
I hope someone will announce this great achievement to the Fedora
community too, and update the following page too.
https://fedoraproject.org/wiki/SIGs/PyTorch/packagingStatus
Yes, this is nice that we have pytorch in Fedora. Looking at the specfile...
USE_CUDA=OFF
USE_ROCM=OFF
Which does not align with:
%description
PyTorch is a Python package that provides two high-level features:
* Tensor computation (like NumPy) with strong GPU acceleration
GPU acceleration? Also,
USE_OPENMP=OFF
So, no threading? What about at least enabling BLAS? Maybe it is by default.
Not seeing it in the specfile. Without a CUDA version of this, it can't be
used the way it was meant to be. We still need to use pip install to get an
accelerated version:
pip install torch
python3
>> import torch
>> torch.__config__.show()
The config listed there should be compared with the config in the spec file to
get as close to the expected feature set as possible so that people can just
switch. This is a positive step and I would love to switch one day.
Best Regards,
-Steve
3:47 p.m.
On Friday, 08 December 2023 at 16:34, Steve Grubb wrote:
On Friday, December 8, 2023 12:41:59 AM EST Jun Aruga (he / him)
wrote:
> Congratulations for the PyTorch package!
> https://src.fedoraproject.org/rpms/python-torch
>
> I hope someone will announce this great achievement to the Fedora
> community too, and update the following page too.
> https://fedoraproject.org/wiki/SIGs/PyTorch/packagingStatus
Yes, this is nice that we have pytorch in Fedora. Looking at the specfile...
USE_CUDA=OFF
USE_ROCM=OFF
Which does not align with:
%description
PyTorch is a Python package that provides two high-level features:
* Tensor computation (like NumPy) with strong GPU acceleration
GPU acceleration?
Indeed. We have ROCM in Fedora already:
$ sudo dnf list rocm\*devel
Last metadata expiration check: 0:00:24 ago on Sat 09 Dec 2023 22:42:19.
Available Packages
rocm-comgr-devel.x86_64 16.1-2.fc38
updates
rocm-hip-devel.x86_64 5.5.1-10.fc38
updates
rocm-opencl-devel.x86_64 5.5.1-10.fc38
updates
rocm-runtime-devel.x86_64 5.5.0-2.fc38
updates
rocm-smi-devel.x86_64 5.5.1-3.fc38
updates
At least the OpenCL part works quite well for me.
[...]
pip install torch
python3
>>> import torch
>>> torch.__config__.show()
The config listed there should be compared with the config in the spec file to
get as close to the expected feature set as possible so that people can just
switch. This is a positive step and I would love to switch one day.
Good idea.
Regards,
Dominik
--
Fedora https://fedoraproject.org
There should be a science of discontent. People need hard times and
oppression to develop psychic muscles.
-- from "Collected Sayings of Muad'Dib" by the Princess Irulan
4:08 a.m.
On 12/9/23 14:47, Dominik 'Rathann' Mierzejewski wrote:
On Friday, 08 December 2023 at 16:34, Steve Grubb wrote:
> On Friday, December 8, 2023 12:41:59 AM EST Jun Aruga (he / him) wrote:
>> Congratulations for the PyTorch package!
>> https://src.fedoraproject.org/rpms/python-torch
>>
>> I hope someone will announce this great achievement to the Fedora
>> community too, and update the following page too.
>> https://fedoraproject.org/wiki/SIGs/PyTorch/packagingStatus
>
> Yes, this is nice that we have pytorch in Fedora. Looking at the specfile...
>
> USE_CUDA=OFF
> USE_ROCM=OFF
>
> Which does not align with:
>
> %description
> PyTorch is a Python package that provides two high-level features:
> * Tensor computation (like NumPy) with strong GPU acceleration
>
> GPU acceleration?
Indeed. We have ROCM in Fedora already:
$ sudo dnf list rocm\*devel
Last metadata expiration check: 0:00:24 ago on Sat 09 Dec 2023 22:42:19.
Available Packages
rocm-comgr-devel.x86_64 16.1-2.fc38
updates
rocm-hip-devel.x86_64 5.5.1-10.fc38
updates
rocm-opencl-devel.x86_64 5.5.1-10.fc38
updates
rocm-runtime-devel.x86_64 5.5.0-2.fc38
updates
rocm-smi-devel.x86_64 5.5.1-3.fc38
updates
At least the OpenCL part works quite well for me.
We have made a lot of progress towards getting ROCm packaged in Fedora but we're not
quite done and I know there are a few yet-unpackaged ROCm components which are required
before ROCm support can be enabled for pytorch.
The current status list of ROCm packages is in the wiki:
https://fedoraproject.org/wiki/SIGs/HC#Package_status
The current plan is to have ROCm 6.0.0 in F40 before branch, assuming that the release
dates line up. As far as I know, AMD has not given an exact release date for 6.0 but ROCm
6.0 was announced last week and it seems likely to release well before F40 branch.
Tim
> [...]
>> pip install torch
>> python3
>>>>> import torch
>>>>> torch.__config__.show()
>>
>> The config listed there should be compared with the config in the spec file to
>> get as close to the expected feature set as possible so that people can just
>> switch. This is a positive step and I would love to switch one day.
>
> Good idea.
>
> Regards,
> Dominik
3:59 a.m.
On 12/8/23 08:34, Steve Grubb wrote:
On Friday, December 8, 2023 12:41:59 AM EST Jun Aruga (he / him)
wrote:
> Congratulations for the PyTorch package!
> https://src.fedoraproject.org/rpms/python-torch
>
> I hope someone will announce this great achievement to the Fedora
> community too, and update the following page too.
> https://fedoraproject.org/wiki/SIGs/PyTorch/packagingStatus
Yes, this is nice that we have pytorch in Fedora. Looking at the specfile...
USE_CUDA=OFF
USE_ROCM=OFF
Which does not align with:
%description
PyTorch is a Python package that provides two high-level features:
* Tensor computation (like NumPy) with strong GPU acceleration
GPU acceleration? Also,
GPU acceleration is not enabled for the pytorch packages and that is intentional, for now.
pytorch has a mess of third party dependencies which are managed upstream using git
subrepos that point to external dependencies that may or may not be easy to package for
Fedora. From the beginning, our plan has been to get pytorch packaged for CPU only first
and add accelerator support as we can. Perhaps the description for pytorch needs to be
changed but our intent is to enable ROCm support for F40.
I don't have the exact list of packages remaining before we can enable ROCm support
for pytorch in front of me but I believe that we're down into the single digits and
the biggest hurdle at the moment is ROCm's miopen due to some incompatibility with
Fedora's llvm or hipcc.
USE_OPENMP=OFF
So, no threading? What about at least enabling BLAS? Maybe it is by default.
Not seeing it in the specfile. Without a CUDA version of this, it can't be
used the way it was meant to be. We still need to use pip install to get an
accelerated version:
I'm not familiar with OpenMP or what might be required there, Tom (cc'd) would
know more on that exact detail.
I doubt that a CUDA version of pytorch will ever be packagable for the Fedora repos - the
licensing on CUDA would have to change before that happens and while it's possible, it
doesn't seem likely in the foreseeable future.
It would be great to enable support for Intel accelerators but that is a different project
for a different day. ROCm is the only accelerator support that we have scoped out at this
point.
pip install torch
python3
>>> import torch
>>> torch.__config__.show()
The config listed there should be compared with the config in the spec file to
get as close to the expected feature set as possible so that people can just
switch. This is a positive step and I would love to switch one day.
In general, there are two reasons why a torch feature is not enabled in the Fedora
package:
1. The license of a dependency for that feature is incompatible with Fedora
2. One or more dependencies are not yet packaged for Fedora
Obviously, features that fall into (1) are very difficult, if not impossible for us to
work around. Features that fall into (2) will likely need more time - the first build for
PyTorch was about a week ago and we still have work to do.
We are working to get the pytorch packages in Fedora to be as complete as we can make
them. If anyone is interested in helping, please join us on discourse (#ai-ml-sig) or
Matrix (#ai-ml:fedoraproject.org).
Tim
> Best Regards,
> -Steve
>
> --
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
11:04 a.m.
On Monday, December 11, 2023 4:59:45 AM EST Tim Flink wrote:
On 12/8/23 08:34, Steve Grubb wrote:
> On Friday, December 8, 2023 12:41:59 AM EST Jun Aruga (he / him) wrote:
>
>> Congratulations for the PyTorch package!
>> https://src.fedoraproject.org/rpms/python-torch
>>
>> I hope someone will announce this great achievement to the Fedora
>> community too, and update the following page too.
>> https://fedoraproject.org/wiki/SIGs/PyTorch/packagingStatus
>
>
> Yes, this is nice that we have pytorch in Fedora. Looking at the
> specfile...
>
> USE_CUDA=OFF
> USE_ROCM=OFF
>
> Which does not align with:
>
> %description
> PyTorch is a Python package that provides two high-level features:
> * Tensor computation (like NumPy) with strong GPU acceleration
>
>
> GPU acceleration? Also,
GPU acceleration is not enabled for the pytorch packages and that is
intentional, for now. pytorch has a mess of third party dependencies which
are managed upstream using git subrepos that point to external
dependencies that may or may not be easy to package for Fedora.
Yes, I am familiar with the orignal LUA version which I had to build locally.
From the beginning, our plan has been to get pytorch packaged for CPU
only
first and add accelerator support as we can. Perhaps the description for
pytorch needs to be changed but our intent is to enable ROCm support for
F40.
If you are doing CPU only, you really should enable a BLAS backend. Fedora
has flexiblas available.
I don't have the exact list of packages remaining before we can
enable ROCm
support for pytorch in front of me but I believe that we're down into the
single digits and the biggest hurdle at the moment is ROCm's miopen due to
some incompatibility with Fedora's llvm or hipcc.
> USE_OPENMP=OFF
>
> So, no threading? What about at least enabling BLAS? Maybe it is by
> default. Not seeing it in the specfile. Without a CUDA version of this,
> it can't be used the way it was meant to be. We still need to use pip
> install to get an accelerated version:
I'm not familiar with OpenMP or what might be required there, Tom (cc'd)
would know more on that exact detail.
GCC should natively support it - unless it uses something brand new GCC
hasn't adopted yet.
I doubt that a CUDA version of pytorch will ever be packagable for
the
Fedora repos - the licensing on CUDA would have to change before that
happens and while it's possible, it doesn't seem likely in the foreseeable
future.
It would be great to enable support for Intel accelerators but that
is a
different project for a different day. ROCm is the only accelerator
support that we have scoped out at this point.
> pip install torch
> python3
>
>>>> import torch
>>>> torch.__config__.show()
>
>
> The config listed there should be compared with the config in the spec
> file to get as close to the expected feature set as possible so that
> people can just switch. This is a positive step and I would love to
> switch one day.
In general, there are two reasons why a torch feature is not enabled in the
Fedora package:
1. The license of a dependency for that feature is incompatible with
Fedora
2. One or more dependencies are not yet packaged for Fedora
I think you can add OpenMP and BLAS support easily. That should be a small
win.
-Steve
Obviously, features that fall into (1) are very difficult, if not
impossible for us to work around. Features that fall into (2) will likely
need more time - the first build for PyTorch was about a week ago and we
still have work to do.
We are working to get the pytorch packages in Fedora to be as
complete as
we can make them. If anyone is interested in helping, please join us on
discourse (#ai-ml-sig) or Matrix (#ai-ml:fedoraproject.org).
Tim
> Best Regards,
> -Steve
>
> --
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List
> Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines List
> Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.
> org Do not reply to spam, report it:
> https://pagure.io/fedora-infrastructure/new_issue
--
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List
Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List
Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.or
g Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
7:46 a.m.
On 12/8/23 7:34 AM, Steve Grubb wrote:
On Friday, December 8, 2023 12:41:59 AM EST Jun Aruga (he / him)
wrote:
> Congratulations for the PyTorch package!
> https://src.fedoraproject.org/rpms/python-torch
>
> I hope someone will announce this great achievement to the Fedora
> community too, and update the following page too.
> https://fedoraproject.org/wiki/SIGs/PyTorch/packagingStatus
Yes, this is nice that we have pytorch in Fedora. Looking at the specfile...
USE_CUDA=OFF
USE_ROCM=OFF
Which does not align with:
%description
PyTorch is a Python package that provides two high-level features:
* Tensor computation (like NumPy) with strong GPU acceleration
GPU acceleration? Also,
USE_OPENMP=OFF
So, no threading? What about at least enabling BLAS? Maybe it is by default.
Not seeing it in the specfile. Without a CUDA version of this, it can't be
used the way it was meant to be. We still need to use pip install to get an
accelerated version:
pip install torch
python3
I made the choice to focus on doing cpu first believing it would be a
good base to build out from.
To be useful, PyTorch needs to be accelerated, but acceleration is
complicated. The acceleration stack as well as the hw drivers must be
in Fedora. This work is tracked on the HC sig page. Unfortunately CUDA
is not open source. But ROCm is. There is a parallel effort as Tim has
said to enable the ROCm packages that PyTorch needs. Most of them are
in Fedora, but we are missing a couple of them.
Tom
>>> import torch
>>> torch.__config__.show()
The config listed there should be compared with the config in the spec file to
get as close to the expected feature set as possible so that people can just
switch. This is a positive step and I would love to switch one day.
Best Regards,
-Steve
--
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
3:42 a.m.
On 12/7/23 22:41, Jun Aruga (he / him) wrote:
Congratulations for the PyTorch package!
https://src.fedoraproject.org/rpms/python-torch
I hope someone will announce this great achievement to the Fedora
community too, and update the following page too.
https://fedoraproject.org/wiki/SIGs/PyTorch/packagingStatus
PyTorch packages in Fedora are still a work in progress - torchvision and torchdata are
still in review, for example. We are definitely planning to announce it once we've
made more progress and are also planning to make a F40 change proposal for PyTorch.
I didn't realize that the dependency packaging list was out of date but I'll
review it this week and make sure it has been updated.
Tim
> Jun
>
> On Wed, Oct 11, 2023 at 11:55 PM Kaitlyn Abdo <kabdo(a)redhat.com> wrote:
>>
>> Hello everyone,
>>
>> I hope this email finds everyone well. We are currently packaging PyTorch for
Fedora and we are actively packaging the dependencies. We’re reaching out to see if anyone
is interested in joining the project. If you’re interested in more information, our
meeting notes are in a GitHub repo and we have the AI/ML SIG discussion page where we’ve
hosted all of our post meeting discussions. We have a WikiPage for our package assignment
list if you would like to see what needs packaging or just to see the status of the
project. Our next meeting is tomorrow, October 11th at 9AM EST and it’s on the FedoCal
under the SIG calendar. All lists are found below. If you’re interested in joining, have
any questions, or want to get a calendar invite for our reoccurring meeting, please feel
free to email me at kabdo(a)redhat.com or Teng at tema(a)redhat.com.
>>
>> AI/ML SIG Discussion Page: https://discussion.fedoraproject.org/tag/ai-ml-sig
>> Meeting Notes: https://github.com/kaitlynabdo/pytorch-fedora-meeting-notes
>> Packaging Assignment List:
https://fedoraproject.org/wiki/SIGs/PyTorch/packagingStatus
>> FedoCal: https://calendar.fedoraproject.org/SIGs/
>>
>> Thank you!
>> Kaitlyn Abdo
>> _______________________________________________
>> devel mailing list -- devel(a)lists.fedoraproject.org
>> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>> Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
>
>
>
149
days inactive
210
days old
8 comments
6 participants
participants (6)
-
Dominik 'Rathann' Mierzejewski -
Jun Aruga (he / him) -
Kaitlyn Abdo -
Steve Grubb -
Tim Flink -
Tom Rix