On Tue, Aug 27, 2019 at 4:07 AM Lumir Balhar <lbalhar(a)redhat.com> wrote:
On 8/26/19 9:19 PM, Michael McCune wrote:
> On Mon, Aug 26, 2019 at 2:35 PM Benson Muite <benson_muite(a)emailplus.org>
wrote:
>>
>> On 8/26/19 8:42 PM, Michael McCune wrote:
>>> On Mon, Aug 26, 2019 at 3:10 AM Benson Muite
<benson_muite(a)emailplus.org> wrote:
>>>> On 8/22/19 3:57 PM, Michael McCune wrote:
>>>>> On Thu, Aug 22, 2019 at 3:37 AM Lumir Balhar
<lbalhar(a)redhat.com> wrote:
>>>>>> Hello.
>>>>>>
>>>>>> On 8/21/19 3:37 PM, Michael McCune wrote:
>>>>>>> hi all,
>>>>>>>
>>>>>>> i am writing today to get a sense for how the group feels
about
>>>>>>> creating content that focuses on containerized machine
learning
>>>>>>> workflows on fedora.
>>>>>>>
>>>>>>> i have been creating cloud native ml applications for a few
years now
>>>>>>> and most of my work begins on fedora inside of containers.
from the
>>>>>>> wiki page[0], it seems apparent that one of the major
focuses of this
>>>>>>> group is around packaging various frameworks and tools for
fedora, but
>>>>>>> i am curious how the sig feels about embracing a more
container
>>>>>>> focused approach to machine learning development on fedora.
>>>>>>>
>>>>>>> i think the initial outputs from a push like this could
easily see us
>>>>>>> creating some tutorial content and fedora based images for
doing work
>>>>>>> that uses modern machine learning frameworks (eg spark,
tensorflow,
>>>>>>> etc). i also think that focusing on containers gives us an
easier way
>>>>>>> to tell stories about machine learning on silverblue and
fedora
>>>>>>> coreos.
>>>>>>>
>>>>>>> with that said, what does the group think about this?
>>>>>> I think that this is a great idea.
>>>>>>
>>>>>> One of our goals is to gather success stories about how the
Fedora is
>>>>>> used in AI/ML industry and write some blog posts about it with a
"hello
>>>>>> world" examples using the tools already available in
Fedora.
>>>>>>
>>>>> so, i guess what i am proposing are stories about how a fedora user
>>>>> can unlock new power through the use of podman and buildah ;)
>>>> This seems nice
>>>>>> If we identify that some of the working pipelines make sense
also as a
>>>>>> containerized Fedora instance we can definitely create and
maintain a
>>>>>> ready-to-use container and use that also in blog posts mentioned
before
>>>>>> to simplify things and/or offer a different approach how to get
it working.
>>>>>>
>>>>> a "ready-to-use" container sounds like an interesting
idea, i'd love
>>>>> to hear more thoughts about that.
>>>> Would machine translation pipelines also be worth including? Fedora
>>>> Translations are expected to move to use Weblate (
https://weblate.org/)
>>>> which can be integrated with various machine translation engines,
>>>>
http://opennmt.net/,
https://github.com/facebookresearch/UnsupervisedMT,
>>>>
https://github.com/moses-smt/mosesdecoder,
http://thumt.thunlp.org/,
>>>>
https://marian-nmt.github.io/
>>> i think this definitely sounds like an interesting use case.
>>>
>>> imo, we could easily show off several different types of workflows and
>>> uses that all utilize fedora and the container ecosystem for the work.
>>> i think it would be great to have some articles about each of these
>>> different techniques, with examples that a user could try out. i'm
>>> just not sure where we would put these articles and if the ml-sig is
>>> the proper organization to drive them forward, but it seems like
>>> something in our purview. fair warning, i am new to the group here =)
>> Ok, this may also be of interest:
>>
>>
https://www.paddlepaddle.org.cn/documentation/docs/en/1.4/beginners_guide...
>>
>> Note that documentation on installation in Mandarin using Docker is a
>> little bit more comprehensive than in English:
>>
>>
http://en.paddlepaddle.org/documentation/docs/zh/1.5/beginners_guide/inst...
>>
> when we start talking about containers in fedora and workflows, i
> think we should be looking for opportunities to show how podman and
> buildah work and how they make life better on fedora.
I think that we (ml-sig) should start with something simple. I can
imagine an article about some well-known AI/ML hello-world problem to
show that in Fedora we have RPMs ready for AI/ML engineers. It might
happen that we also identify that we are not ready for some centrain
worlflows so we can fix it right away.
i question whether we need to start with rpm based content. from my
experience the rpm's in fedora are outpaced by software that i can use
in containers, additionally most of the software that i have been
using will likely never be packaged as an rpm. i can demonstrate a
hello world in apache spark using containers in a pretty small
article.
Then, when we identify something more complex - for example with
Tensorflow which cannot be installed from RPM - we can write a longer
article where we show how to install Tensorflow from wheels and how to
use it (again some hellow world example is enough). And in an more
complex example like this, we can show how to use buildah/podman and/or
we can create an image for a workflow described in an article if it is
common enought that it'd make sense to maintain that image.
i really don't feel that the container based solutions are necessarily
any more complex than using an rpm and they are certainly less complex
than a user building their own wheel files or rpms.
If you agree, I'd start a new thread to gather some examples we
can
cover in articles.
i definitely agree that we should generate some content, my question
is should be organizing how this is done or is there a way for us each
to contribute at our own pace?
Also, I think that the best platform for this is
https://fedoramagazine.org/
very cool!
peace o/
> >
> > no slam on docker, but these are the type of instructions we could
> > make more fedora-ish by using the native open source tools. thanks for
> > the links!
> >
> > peace o/
> >
> >>>>>> We (Python maintenance team) already did a similar thing
with
> >>>>>> fedora-python-tox:
https://github.com/fedora-python/fedora-python-tox
> >>>>>>
> >>>>> i will check this out
> >>>>>
> >>>>>> Do you know about some good examples we can start with?
> >>>>>>
> >>>>> indeed, i do. i contribute to a community (radanalytics.io)
that
> >>>>> focuses on machine learning on openshift. we containerize tools
like
> >>>>> apache spark, and then demonstrate how to create application
> >>>>> pipelines. much of the spark work i do begins on fedora using
these
> >>>>> containers from the community. although spark is not packaged
for
> >>>>> fedora (for good reason), we can still utilize it by running
the
> >>>>> community images (centos based).
> >>>>>
> >>>>> if this style of workflow is useful to the fedora community, i
can
> >>>>> definitely write some fedora-specific blogs about how users can
run
> >>>>> spark-based machine learning workflows. i guess my larger
question is,
> >>>>> is it appropriate for us to demonstrate workflows that use tools
which
> >>>>> aren't packaged for fedora?
> >>>>>
> >>>>> peace o/
> >>>>>
> >>>>>> Lumír
> >>>>>>> peace o/
> >>>>>>>
> >>>>>>> [0]
https://fedoraproject.org/wiki/SIGs/ML
> >>>>>>> _______________________________________________
> >>>>>>> ml mailing list -- ml(a)lists.fedoraproject.org
> >>>>>>> To unsubscribe send an email to
ml-leave(a)lists.fedoraproject.org
> >>>>>>> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> >>>>>>> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> >>>>>>> List Archives:
https://lists.fedoraproject.org/archives/list/ml@lists.fedoraproject.org
> >>>>>> _______________________________________________
> >>>>>> ml mailing list -- ml(a)lists.fedoraproject.org
> >>>>>> To unsubscribe send an email to
ml-leave(a)lists.fedoraproject.org
> >>>>>> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> >>>>>> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> >>>>>> List Archives:
https://lists.fedoraproject.org/archives/list/ml@lists.fedoraproject.org
> >>>>> _______________________________________________
> >>>>> ml mailing list -- ml(a)lists.fedoraproject.org
> >>>>> To unsubscribe send an email to
ml-leave(a)lists.fedoraproject.org
> >>>>> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> >>>>> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> >>>>> List Archives:
https://lists.fedoraproject.org/archives/list/ml@lists.fedoraproject.org
> >>>> _______________________________________________
> >>>> ml mailing list -- ml(a)lists.fedoraproject.org
> >>>> To unsubscribe send an email to ml-leave(a)lists.fedoraproject.org
> >>>> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> >>>> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> >>>> List Archives:
https://lists.fedoraproject.org/archives/list/ml@lists.fedoraproject.org
> >>> _______________________________________________
> >>> ml mailing list -- ml(a)lists.fedoraproject.org
> >>> To unsubscribe send an email to ml-leave(a)lists.fedoraproject.org
> >>> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> >>> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> >>> List Archives:
https://lists.fedoraproject.org/archives/list/ml@lists.fedoraproject.org
> >> _______________________________________________
> >> ml mailing list -- ml(a)lists.fedoraproject.org
> >> To unsubscribe send an email to ml-leave(a)lists.fedoraproject.org
> >> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> >> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> >> List Archives:
https://lists.fedoraproject.org/archives/list/ml@lists.fedoraproject.org
> > _______________________________________________
> > ml mailing list -- ml(a)lists.fedoraproject.org
> > To unsubscribe send an email to ml-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
https://lists.fedoraproject.org/archives/list/ml@lists.fedoraproject.org
> _______________________________________________
> ml mailing list -- ml(a)lists.fedoraproject.org
> To unsubscribe send an email to ml-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/ml@lists.fedoraproject.org