Hi All, Fedora 33 curl-7.76.1-12.fc34.x86_64 This page show something different in Firefox and Vivaldi that I can get with curl.

Looking at eraseme.html, it look like I am only downloading the left column and not the contents on the right side of the page

What am I doing wrong?

Any way to get git to show me the revision without downloading the turkey?

ToddAndMargo via users writes: > On 10/10/21 17:14, Sam Varshavchik wrote: >> You're trying to reinvent the wheel. This a web interface to a plain, >> garden variety, git repository. >> >> Forget curl. Use git. >> >> $ git clone https://gitlab.freedesktop.org/spice/win32/spice-nsis.git >> Cloning into 'spice-nsis'… >> remote: Enumerating objects: 461, done. >> remote: Total 461 (delta 0), reused 0 (delta 0), pack-reused 461 >> Receiving objects: 100% (461/461), 89.54 KiB | 8.14 MiB/s, done. >> Resolving deltas: 100% (275/275), done. >> $ cd spice-nsis >> $ git show | sed '1,10p' >> commit f6ad44f35f5caeec51b7002169977272d85701a3 >> commit f6ad44f35f5caeec51b7002169977272d85701a3 >> Author: Sandro Bonazzola <sbonazzo(a)redhat.com&gt; >> Author: Sandro Bonazzola <sbonazzo(a)redhat.com&gt; >> Date:   Tue Feb 5 16:49:45 2019 +0100 >> Date:   Tue Feb 5 16:49:45 2019 +0100 >> >> >>     virtio-win: rebase on 0.164 >>     virtio-win: rebase on 0.164 >> >> And a little bit of scripting, or by picking the right options to >> "git show", to get just what you want. I'm too lazy to read >> git-show's man page, but there's probably a way to have it show just >> the commit message, and nothing else. >> >> Anytime you want to recheck, just cd back to the same directory, do a >> "git pull", and see what you got. > > > Any way to get git to show me the revision > without downloading the turkey? Looks like this entire git repo is only 348kb. Now, with the linux-kernel repo this would probably be a consideration but not for something as small as this.

ToddAndMargo via users writes: > On 10/10/21 18:08, Sam Varshavchik wrote: >> >> Looks like this entire git repo is only 348kb. Now, with the >> linux-kernel repo this would probably be a consideration but not for >> something as small as this. > > I am being a perfectionist.  I  will drop back to this > if I can't figure out a workaround Any kind of a web-scraping approach is going to be vulnerable to any UI or HTML change that github might drop, unexpectedly. And this may not even be a visible UI change. git has a much stable interface, and gives you the metadata pretty much as is. And, again: git clone needs to be done only once. Only git pull is needed every time, and that would only download any changes.

ToddAndMargo via users writes: > Hi All, > > Fedora 33 > curl-7.76.1-12.fc34.x86_64 > > This page show something different in Firefox and Vivaldi > that I can get with curl. It's quite common for a web site to inspect the headers from the web client, digest what's in the header that's sent by the client which advertise what kind of content the client claims to understand, then send the appropriate content. So the same URL can return completely different results to curl, and to a full-featured web browser. > Looking at eraseme.html, it look like I am only > downloading the left column and not the contents > on the right side of the page Another possibility is that the web page consists of different frames, and a separate request is needed for each frame. > What am I doing wrong? You're trying to reinvent the wheel. This a web interface to a plain, garden variety, git repository. Forget curl. Use git. $ git clone https://gitlab.freedesktop.org/spice/win32/spice-nsis.git Cloning into 'spice-nsis'… remote: Enumerating objects: 461, done. remote: Total 461 (delta 0), reused 0 (delta 0), pack-reused 461 Receiving objects: 100% (461/461), 89.54 KiB | 8.14 MiB/s, done. Resolving deltas: 100% (275/275), done. $ cd spice-nsis $ git show | sed '1,10p' commit f6ad44f35f5caeec51b7002169977272d85701a3 commit f6ad44f35f5caeec51b7002169977272d85701a3 Author: Sandro Bonazzola <sbonazzo(a)redhat.com&gt; Author: Sandro Bonazzola <sbonazzo(a)redhat.com&gt; Date:   Tue Feb 5 16:49:45 2019 +0100 Date:   Tue Feb 5 16:49:45 2019 +0100    virtio-win: rebase on 0.164    virtio-win: rebase on 0.164 And a little bit of scripting, or by picking the right options to "git show", to get just what you want. I'm too lazy to read git-show's man page, but there's probably a way to have it show just the commit message, and nothing else. Anytime you want to recheck, just cd back to the same directory, do a "git pull", and see what you got.

This simulates what I came up with in my actual code: $ curl --silent 'https://gitlab.freedesktop.org/spice/win32/spice-nsis/-/commits/master' | raku -ne 'my Str $x=slurp(); $x~~s/.*? "rebase on " //; $x~~s/ ("</a>") .* //; print "$x\n";' 0.164

I wrote a program to go out to web pages and check revisions on about 70 programs that I support at my customer sites.

With a web browser, you can see the revision https://gitlab.freedesktop.org/spice/win32/spice-nsis/-/tree/master but right pane is dynamically generated

On 10/12/21 02:13, ToddAndMargo via users wrote: > > I wrote a program to go out to web pages and check > revisions on about 70 programs that I support at > my customer sites. Well, I have a couple of questions...  The commands and output that you posted earlier pulled some text from a commit message, which isn't a revision at all.  It's neither a commit ID, nor a tag, nor a release. It's not even guaranteed to be unique.  So, specifically what are you looking for? For most projects, I'd suggest using the API to get the most recent release or tag, since they don't release on every commit (and I'm guessing that you're not deploying directly from git on each of their commits either, but maybe you are).  However, spice-nsis doesn't appear to have any of either of those. So, maybe you mean you're looking for the last commit ID.  If so, you can use the "curl | jq" commands that I posted earlier to get those. > With a web browser, you can see the revision > https://gitlab.freedesktop.org/spice/win32/spice-nsis/-/tree/master > but right pane is dynamically generated Well, the whole page is dynamically generated.  It's more accurate and meaningful to say that the commit message and commit ID (the revision) come from the API, which you can query with curl and jq.

ToddAndMargo via users wrote: > On 10/12/21 21:55, Gordon Messmer wrote: >> Well, the whole page is dynamically generated.  It's more accurate and >> meaningful to say that the commit message and commit ID (the revision) >> come from the API, which you can query with curl and jq. > > Oh ya. No fooling. The way OI am getting the revison of > the program is the worst way of doing it. > > And I have no choice: > > Spice Guest Tools Downloads are not versioned > https://gitlab.freedesktop.org/spice/win32/spice-nsis/-/issues/6 These are both the same binary: https://www.spice-space.org/download/windows/spice-guest-tools/spice-gues... https://www.spice-space.org/download/windows/spice-guest-tools/spice-gues... So the number you are getting from scraping the commit message (the harder way) isn't even accurate as far as I can tell. It seems like you're asking for help solving the wrong problem (or, at least, a harder problem than necessary). And even when folks suggest better ways to solve that problem --using a proper API or calling git directly-- you're still doing it the more fragile way in the end. I'd be inclined to suggest taking a step back and try to describe the actual end goal and determine if the problem you're trying to solve is even the right one to solve. :) Whether that ends up being something relevant to this list or not, I don't know.