Zulip Chat Archive

Stream: general

Topic: Download speed from new olean storage on Azure


Gabriel Ebner (Feb 24 2020 at 10:31):

So I just tried out Patrick's leancli tool. But it is horribly slow here at VU. It takes 17 seconds to download an olean tarball, compared to 3 seconds at home. Does anybody know what's going on here? (I've also added a speed comparison at two other locations in Vienna, and it's pretty slow there as well.)

# at VU:
$ curl https://oleanstorage.blob.core.windows.net/mathlib/6f3f977cbb4e6c3b65e8f8ceafb5c50dc67ddf50.tar.gz >/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21.8M  100 21.8M    0     0  1374k      0  0:00:16  0:00:16 --:--:-- 1537k
# at home (ziggo in Amstelveen):
$ curl https://oleanstorage.blob.core.windows.net/mathlib/6f3f977cbb4e6c3b65e8f8ceafb5c50dc67ddf50.tar.gz >/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21.8M  100 21.8M    0     0  6451k      0  0:00:03  0:00:03 --:--:-- 6449k
# at TU Wien:
$ curl https://oleanstorage.blob.core.windows.net/mathlib/6f3f977cbb4e6c3b65e8f8ceafb5c50dc67ddf50.tar.gz >/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21.8M  100 21.8M    0     0  3093k      0  0:00:07  0:00:07 --:--:-- 2959k
# in Vienna (UPC):
curl https://oleanstorage.blob.core.windows.net/mathlib/6f3f977cbb4e6c3b65e8f8ceafb5c50dc67ddf50.tar.gz >/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21.8M  100 21.8M    0     0  2884k      0  0:00:07  0:00:07 --:--:-- 3123k

Rob Lewis (Feb 24 2020 at 14:11):

I get the same speed as you at the VU.

Rob Lewis (Feb 24 2020 at 14:16):

I see a few threads about this on Google with no solutions. Could we get a few more data points from people running curl https://oleanstorage.blob.core.windows.net/mathlib/6f3f977cbb4e6c3b65e8f8ceafb5c50dc67ddf50.tar.gz >/dev/null?

Anne Baanen (Feb 24 2020 at 14:26):

From my computer at home (T-Mobile Thuis, Zeist; 5 MB/s typical speed):

$ curl https://oleanstorage.blob.core.windows.net/mathlib/6f3f977cbb4e6c3b65e8f8ceafb5c50dc67ddf50.tar.gz >/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21.8M  100 21.8M    0     0   719k      0  0:00:31  0:00:31 --:--:--  856k

Sebastian Ullrich (Feb 24 2020 at 14:33):

KIT

curl https://oleanstorage.blob.core.windows.net/mathlib/6f3f977cbb4e6c3b65e8f8ceafb5c50dc67ddf50.tar.gz >/dev/null

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21.8M  100 21.8M    0     0  2689k      0  0:00:08  0:00:08 --:--:-- 3111k

compare to

curl -L https://github.com/leanprover/lean/releases/download/v3.4.2/lean-3.4.2-linux.tar.gz >/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   613  100   613    0     0  14951      0 --:--:-- --:--:-- --:--:-- 14951
100 14.5M  100 14.5M    0     0  7038k      0  0:00:02  0:00:02 --:--:-- 7983k

Rob Lewis (Feb 24 2020 at 15:14):

Okay, apparently we had to set up a CDN to do this properly. New format for the URLs: https://oleanstorage.azureedge.net/mathlib/

Rob Lewis (Feb 24 2020 at 15:15):

e.g. curl https://oleanstorage.azureedge.net/mathlib/6f3f977cbb4e6c3b65e8f8ceafb5c50dc67ddf50.tar.gz >/dev/null should be much faster.

Rob Lewis (Feb 24 2020 at 15:15):

cc @Patrick Massot for your script

Sebastian Ullrich (Feb 24 2020 at 15:21):

20x as fast :thumbs_up:


Last updated: Dec 20 2023 at 11:08 UTC