Zulip Chat Archive

Stream: general

Topic: automatically generate ProjectName.lean


Kevin Buzzard (Mar 24 2024 at 11:25):

In Lean 3 there was a straightforward command (I don't remember but it might have been lean --make) which would just build all the files in src. This default functionality would be all that I'd ever need with a mathematics project.

Now we have a different set-up whereby whenever I add a new file to mathlib I have to edit Mathlib.lean. I don't want to go through all this with FLT.lean. So what do I want to do here? I suspect I need some kind of "trigger" which, whenever I e.g. push or commit or whatever triggers this, something should go through the FLT directory and create FLT.lean based on the current contents of the FLT directory. What should I be doing here?

Yaël Dillies (Mar 24 2024 at 11:27):

I have a script for that in APAP: https://github.com/YaelDillies/LeanAPAP/blob/master/scripts/mk_all.sh

Yaël Dillies (Mar 24 2024 at 11:28):

So instead of running lake build I run ./scripts/mk_all.sh; lake build

Yaël Dillies (Mar 24 2024 at 11:29):

And since I build my project I push commits (because I don't use bors, so I discipline myself to keep the build green), LeanAPAP.lean is always up to date.

Eric Wieser (Mar 24 2024 at 15:24):

You can also just not create FLT.lean at all

Eric Wieser (Mar 24 2024 at 15:25):

It only matters if you expect people to write import FLT rather than working in your repository directly

Rémy Degenne (Mar 24 2024 at 15:29):

I think leanblueprint needs that file in order to check that the lean declarations referenced in the blueprint actually exist in the project.

Eric Wieser (Mar 24 2024 at 15:43):

It would be pretty easy to teach leanblueprint to iterate over FLT/**/*.lean and import everything instead

Eric Wieser (Mar 24 2024 at 15:43):

It's unfortunate that it can't just ask lake to give it the list of files (it's already easy to tell Lake to have this glob behavior)

Kevin Buzzard (Mar 24 2024 at 16:25):

What I want is a way to compile all the files in the project. But now I remember that there's a way to do this in the lakefile.

Michael Stoll (Mar 24 2024 at 17:04):

something like

@[default_target]
lean_lib «EulerProducts» where
  globs := #[.submodules `EulerProducts]
  -- add any library configuration options here

(replace EulerProducts by FLT).

Asei Inoue (Mar 26 2024 at 16:19):

@Kevin Buzzard I have a script for this purpose.

https://github.com/Seasawher/import-all

Yaël Dillies (Mar 29 2024 at 10:37):

Actually, could we have a simple script inside mathlib to do this job? It's kind of annoying to do by hand every time.

Yaël Dillies (Mar 29 2024 at 10:38):

And we could replace the review-dog suggestions (which are often off by one) by a suggestion to use the script.

Damiano Testa (Mar 29 2024 at 16:47):

In some sense, the script is already there, since CI uses it:

(
  cd "$(git rev-parse --show-toplevel)" || exit 1
  for gp in Mathlib MathlibExtras Mathlib/Tactic Counterexamples Archive
  do
   git ls-files "$gp/*.lean" | LC_ALL=C sort | sed 's/\.lean//;s,/,.,g;s/^/import /' > "$gp.lean"
  done
)

Damiano Testa (Mar 29 2024 at 16:49):

To make it a bit more stable, there could be a first step [Edit: added above already] that makes sure to run from the git-root directory (which is guaranteed during CI, but may not be running it locally).

Damiano Testa (Mar 29 2024 at 17:07):

But also, what would be the problem if the step that builds these "global" files did not trigger a CI error if it notices differences?

CI produces the file and builds mathlib against it. The files are in master and you download them when you upgrade mathlib anyway. If anyone wants to get the files synchronized locally, great, but if they are not, it simply means that import Mathlib locally may not mean the same as import Mathlib on master.

Yaël Dillies (Mar 29 2024 at 17:12):

I think the issues are that

  1. If you add a leaf file, CI won't know about it unless you add it to Mathlib (that can be fixed by tweaking the lakefile)
  2. We don't know how to make bors run this "Update Mathlib" once on each batch

Damiano Testa (Mar 29 2024 at 17:18):

It is fine to not change this, but if you create in CI the files they will contain all leaf files and then wouldn't lake build build them as well? Anyway, as I said, I am not trying to push for this change: I do in fact use a script that I run that automatically updates those files anyway.

Yaël Dillies (Mar 29 2024 at 17:20):

Yes, but that means you (= beginners, mostly) won't see the build errors until the PR is handed to bors. Of course this already happens because mathlib keeps on changing, but it's easy to not make it worse (eg by instructing lake build to build all submodules of mathlib, rather than just Mathlib)

Damiano Testa (Mar 29 2024 at 17:22):

Anyway, the code above is essentially the script that will make the "local" files exactly the same as the ones that CI checks. So simply PRing that as a further scripts/mk_all.sh file could work, right?

Yaël Dillies (Mar 29 2024 at 17:22):

Certainly, at least for the short term

Damiano Testa (Mar 29 2024 at 17:27):

#11781

Yaël Dillies (Apr 03 2024 at 16:03):

@Kevin Buzzard, after #11853, you will be able to run lake exe mkAll and your file FLT.lean will be automatically generated

Yaël Dillies (Jun 05 2024 at 11:08):

@Kevin Buzzard, update: It has landed!

Yaël Dillies (Jun 05 2024 at 11:09):

lake exe mk-all in FLT as much as you want

Michael Rothgang (Jun 05 2024 at 11:10):

lake exe mk_all, right?

Ruben Van de Velde (Jun 05 2024 at 11:14):

I'll push a mathlib bump in a little bit

Ruben Van de Velde (Jun 05 2024 at 11:24):

(FLT#96)

Yaël Dillies (Jun 06 2024 at 08:23):

Michael Rothgang said:

lake exe mk_all, right?

Oh yeah, I couldn't call it mk-all for some unknown reason

Damiano Testa (Jun 06 2024 at 09:13):

I think that - is simply not allowed in declaration names. I'm not completely sure why is that. Maybe we could have forced it with \f<<>>?

Floris van Doorn (Jun 07 2024 at 13:17):

I just tried lake exe mk_all for the first time, and I ran into an error. This occurs both in Mathlib and on a downstream project:

$ lake exe mk_all --help
✔ [3/8] Built Mathlib.Util.GetAllModules:c.o
✔ [4/8] Built Cli.Basic
✔ [5/8] Built mk_all
✔ [6/8] Built Cli.Basic:c.o
✔ [7/8] Built mk_all:c.o
✖ [8/8] Building mk_all
trace: .> c:\Users\Floris\.elan\toolchains\leanprover--lean4---v4.9.0-rc1\bin\leanc.exe -o .\.\.lake/packages\mathlib\.lake\build\bin\mk_all.exe .\.\.lake/packages\mathlib\.lake\build\ir\mk_all.c.o.noexport .\.\.lake/packages\Cli\.lake\build\ir\Cli\Basic.c.o.export .\.\.lake/packages\mathlib\.lake\build\ir\Mathlib\Util\GetAllModules.c.o.export -leanshared
info: stderr:
ld.lld: error: undefined symbol: l_Lake_logToStream
>>> referenced by .\.\.lake/packages\mathlib\.lake\build\ir\Mathlib\Util\GetAllModules.c.o.export:(l_getLeanLibs___lambda__1)
>>> referenced by .\.\.lake/packages\mathlib\.lake\build\ir\Mathlib\Util\GetAllModules.c.o.export:(l_getLeanLibs___lambda__1___boxed)

ld.lld: error: undefined symbol: l_Lake_findInstall_x3f
>>> referenced by .\.\.lake/packages\mathlib\.lake\build\ir\Mathlib\Util\GetAllModules.c.o.export:(l_getLeanLibs)

ld.lld: error: undefined symbol: l_Lake_LakeOptions_mkLoadConfig
>>> referenced by .\.\.lake/packages\mathlib\.lake\build\ir\Mathlib\Util\GetAllModules.c.o.export:(l_getLeanLibs)

ld.lld: error: undefined symbol: l_Lake_CliError_toString
>>> referenced by .\.\.lake/packages\mathlib\.lake\build\ir\Mathlib\Util\GetAllModules.c.o.export:(l_getLeanLibs)
>>> referenced by .\.\.lake/packages\mathlib\.lake\build\ir\Mathlib\Util\GetAllModules.c.o.export:(l_getLeanLibs)

ld.lld: error: undefined symbol: l_Lake_OutStream_get
>>> referenced by .\.\.lake/packages\mathlib\.lake\build\ir\Mathlib\Util\GetAllModules.c.o.export:(l_getLeanLibs)

ld.lld: error: undefined symbol: l_Lake_AnsiMode_isEnabled
>>> referenced by .\.\.lake/packages\mathlib\.lake\build\ir\Mathlib\Util\GetAllModules.c.o.export:(l_getLeanLibs)

ld.lld: error: undefined symbol: l_Lake_loadWorkspace
>>> referenced by .\.\.lake/packages\mathlib\.lake\build\ir\Mathlib\Util\GetAllModules.c.o.export:(l_getLeanLibs)

ld.lld: error: undefined symbol: l_Lake_MainM_runLogIO_replay
>>> referenced by .\.\.lake/packages\mathlib\.lake\build\ir\Mathlib\Util\GetAllModules.c.o.export:(l_getLeanLibs)
>>> referenced by .\.\.lake/packages\mathlib\.lake\build\ir\Mathlib\Util\GetAllModules.c.o.export:(l_getLeanLibs)

ld.lld: error: undefined symbol: initialize_Lake_CLI_Main
>>> referenced by .\.\.lake/packages\mathlib\.lake\build\ir\Mathlib\Util\GetAllModules.c.o.export:(initialize_Mathlib_Util_GetAllModules)

ld.lld: error: undefined symbol: l_Lake_defaultConfigFile
>>> referenced by .\.\.lake/packages\mathlib\.lake\build\ir\Mathlib\Util\GetAllModules.c.o.export:(.refptr.l_Lake_defaultConfigFile)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: external command 'c:\Users\Floris\.elan\toolchains\leanprover--lean4---v4.9.0-rc1\bin\leanc.exe' exited with code 1
Some builds logged failures:
- mk_all
error: build failed

Utensil Song (Jun 07 2024 at 13:26):

This looks like your cache is still broken. Did you force fetch cache after reinstalling the Lean toolchain?

Floris van Doorn (Jun 07 2024 at 13:28):

Good point. It might be related to the errors I had earlier. I'll try it fresh now.

Floris van Doorn (Jun 07 2024 at 13:33):

I now tried it again on a copy of Mathlib that I haven't touched earlier today, and I get the same errors, even after running lake exe cache get!. lake build succeeds normally.

Can you check that lake exe mk_all works for you on Mathlib master?

Sebastian Ullrich (Jun 07 2024 at 13:37):

Very few programs import Lake so it's not unthinkable that this is a new, Windows-only issue with linking against Lake

Floris van Doorn (Jun 07 2024 at 13:38):

That was my alternative guess, that it is a Windows-only issue. Let me know if I can provide other information that is useful.

Sébastien Gouëzel (Jun 07 2024 at 13:52):

Same error on master on my (windows) computer.

Damiano Testa (Jun 07 2024 at 13:56):

On Linux the command works for me (and it is also used by CI).

Utensil Song (Jun 07 2024 at 14:15):

I think this might be related to lean4#3601 which has extra requirements to link on Windows.

image.png

Kim Morrison (Jun 10 2024 at 11:09):

Damiano, can we leave it to you to summon whatever help is needed to get this fixed?

Damiano Testa (Jun 10 2024 at 11:16):

I can try, but it would help if someone who uses windows can do some testing, since I do not have any windows computer.

Damiano Testa (Jun 10 2024 at 11:17):

In fact, if someone who uses windows is willing to have a session where we try to work this out during a call, that might be the easiest.

Michael Rothgang (Jun 10 2024 at 12:17):

Wild guess, but: is supportInterpreter := true required for mk_all? (Perhaps the answers is "yes, obviously" and I don't understand that flag well - in which case I'll be happy to be enlightened.)

Utensil Song (Jun 10 2024 at 13:41):

Utensil Song said:

I think this might be related to lean4#3601 which has extra requirements to link on Windows.

image.png

@Floris van Doorn Maybe you can try copying these 2 DLL files into where mk_all is located, and see if it runs?

Floris van Doorn (Jun 10 2024 at 19:44):

If I copy these two files to mathlib/scripts (where mk_all.lean is located), then I get the same error when running lake exe mk_all. I looked into mathlib/.lake/build/bin/ where mk_all.exe is supposed to be(?). Putting the DLL files there gives the same error as well. That folder does not contain mk_all.exe, btw. It does contain cache.exe, shake.exe and mk_all.exe.log.json. The last file contains (roughly) the error message that is output when running lake exe mk_all.

Utensil Song (Jun 11 2024 at 00:33):

The changelog indicates that the DLLs has to be colocated with the exe, to verify if it's the case, we need to find the exe. The fact that it's not there with other built exe is weird. I'll do some more investigation later today on Github Actions Windows CI as I haven't turned on my Windows computer for months.

Floris van Doorn (Jun 12 2024 at 09:10):

I believe the build errors while building the exe, not when executing it.

Utensil Song (Jun 12 2024 at 12:49):

OK, I can reproduce the same build error on Windows CI, while the same works on Ubuntu and Mac CI. And lake env lake exe mk_all can't save it, too (it rescued my other Windows FFI build).

Utensil Song (Jun 12 2024 at 12:59):

The missing symbols are in .elan/toolchains/leanprover--lean4---v4.9.0-rc1/lib/lean/libLake.a

nm -gU libLake.a | grep "l_Lake_logToStream"
0000000000003644 T _l_Lake_logToStream
0000000000003878 T _l_Lake_logToStream___boxed

Judging by the command trace: .> c:\Users\runneradmin\.elan\toolchains\leanprover--lean4---v4.9.0-rc1\bin\leanc.exe -o .\.\.lake/packages\mathlib\.lake\build\bin\mk_all.exe .\.\.lake/packages\mathlib\.lake\build\ir\mk_all.c.o.noexport .\.\.lake/packages\Cli\.lake\build\ir\Cli\Basic.c.o.export .\.\.lake/packages\mathlib\.lake\build\ir\Mathlib\Util\GetAllModules.c.o.export -leanshared

It should be that it misses -lLake ? One example of adding this flag is here.

Utensil Song (Jun 12 2024 at 13:39):

@Floris van Doorn Can you try locally modify .lake/packages/mathlib/lakefile.lean so it looks like

/-- `lake exe mk_all` constructs the files containing all imports for a project. -/
lean_exe mk_all where
  srcDir := "scripts"
  supportInterpreter := true
  weakLinkArgs := #["-lLake"]

and run lake -R exe mk_all?

I have successfully run it on Windows CI.

image.png

Sebastian Ullrich (Jun 12 2024 at 13:42):

Thanks for the investigation, I now see the bug

Floris van Doorn (Jun 12 2024 at 14:44):

Utensil Song said:

Floris van Doorn Can you try locally modify .lake/packages/mathlib/lakefile.lean so it looks like

/-- `lake exe mk_all` constructs the files containing all imports for a project. -/
lean_exe mk_all where
  srcDir := "scripts"
  supportInterpreter := true
  weakLinkArgs := #["-lLake"]

and run lake -R exe mk_all?

I have successfully run it on Windows CI.

image.png

With that it works without error!

One note: it does change the order of the imports in Mathlib.lean, where . is sorted after any letter (I hope that is not platform-dependent?), e.g.:

-import Mathlib.GroupTheory.Order.Min
 import Mathlib.GroupTheory.OrderOfElement
+import Mathlib.GroupTheory.Order.Min

Damiano Testa (Jun 12 2024 at 14:45):

I am so happy that this works! Thank you all for helping debug this issue!

Damiano Testa (Jun 12 2024 at 14:48):

Wrt sorting, the previous shell script specifically used a flag to "uniformize" the sorting. The lake executable uses the docs#System.FilePath.walkDir (or git ls-files '*.lean') and may not do an "internal" sorting of the output of either. Maybe this should be added.

Damiano Testa (Jun 12 2024 at 14:49):

Actually, it does sort internally.

Damiano Testa (Jun 12 2024 at 14:55):

Floris, I am not sure that I am reading your diff correctly, but the webserver sorts the two files as follows:

import Lean.Elab.Command

run_cmd Lean.Elab.Command.liftTermElabM do
  let pair := #["Mathlib.GroupTheory.OrderOfElement", "Mathlib.GroupTheory.Order.Min"]
  guard <| pair  pair.qsort (· < ·)

so ...Min comes earlier.

Floris van Doorn (Jun 12 2024 at 14:59):

After running lake exe mk_all there are modifications to my copy of Mathlib.lean, and it contains

[...]
import Mathlib.GroupTheory.NoncommPiCoprod
import Mathlib.GroupTheory.OrderOfElement
import Mathlib.GroupTheory.Order.Min
import Mathlib.GroupTheory.PGroup
[...]

Floris van Doorn (Jun 12 2024 at 14:59):

Mathlib commit dd36c71ca81 (5 days ago).

Damiano Testa (Jun 12 2024 at 14:59):

Can you try running the code above, to see if it passes the check?

Damiano Testa (Jun 12 2024 at 15:01):

(On my computer with Linux, the test above passes and so does on the webserver.)

Floris van Doorn (Jun 12 2024 at 15:01):

Yes, that code snippet passes without errors.

#eval #["Mathlib.GroupTheory.OrderOfElement", "Mathlib.GroupTheory.Order.Min"].qsort (· < ·)

gives

#["Mathlib.GroupTheory.Order.Min", "Mathlib.GroupTheory.OrderOfElement"]

Damiano Testa (Jun 12 2024 at 15:02):

And this is what I see in Mathlib.lean around where you mention:

import Mathlib.GroupTheory.NoncommCoprod
import Mathlib.GroupTheory.NoncommPiCoprod
import Mathlib.GroupTheory.Order.Min
import Mathlib.GroupTheory.OrderOfElement
import Mathlib.GroupTheory.PGroup

Damiano Testa (Jun 12 2024 at 15:03):

Hmmm

Floris van Doorn (Jun 12 2024 at 15:03):

But we're comparing a System.FilePath coerced to strings in your linked code, and that seems like it would use \ not .

Damiano Testa (Jun 12 2024 at 15:04):

Ah, or / on a Unix system?

Floris van Doorn (Jun 12 2024 at 15:04):

probably!?

Damiano Testa (Jun 12 2024 at 15:06):

This is the ordering for some characters that might be relevant for me:

#eval
  let arr := #[".", "/", "M", "\\"]
  arr == arr.qsort (· < ·)

Floris van Doorn (Jun 12 2024 at 15:09):

That evaluates to true for me, and seems like it would explain the difference :-)

Damiano Testa (Jun 12 2024 at 15:09):

Maybe the comparison should first replace the docs#System.FilePath.pathSeparator with . and then do the sorting.

Damiano Testa (Jun 12 2024 at 15:09):

Let me PR that!

Michael Rothgang (Jun 12 2024 at 16:00):

IIRC, there's some function toModuleName (or similar) in lake, which mk_all also uses for this purpose. (Or did you mean to do this already?)

Damiano Testa (Jun 12 2024 at 16:09):

Are you thinking of docs#moduleNameOfFileName? It is a little inconvenient to use it in the qsort, since the function is in IO, but the sort is pure.

Damiano Testa (Jun 12 2024 at 16:15):

#13777
I tested it locally and it does not change the order of the files on my computer. It also passed the relevant CI step, so CI is also on the same order.

If someone using Windows/MacOS wants to try it out, checking out the branch adomani/sort_all_files and running lake exe mk_all (or lake exe mk_all --git if you have random files in your Mathlib dir) should report No update necessary.

Richard Osborn (Jun 12 2024 at 16:30):

I would map Lean.moduleNameOfFileNameover allModules first and then sort afterwards. It's clunky doing it at the same time.

Damiano Testa (Jun 12 2024 at 16:33):

But then you would have to convert back from Name to FilePath, right?

Damiano Testa (Jun 12 2024 at 16:34):

(and you would need to catch errors, in case the path does not exist, but this is a step that happens later anyway, so it would just be shuffling code around)

Richard Osborn (Jun 12 2024 at 17:35):

Yea, sorry. I didn't see that getAllModules is what is returning the module strings. It would make most sense to sort there. (Not really sure why they are sorting preemptively). The long term solution would be to write a proper sort for System.FilePath.

Michael Rothgang (Jun 14 2024 at 13:05):

Sebastian Ullrich said:

Thanks for the investigation, I now see the bug

Is this bug still present? (Does it need a fix in mathlib?) I'd like to recommend mk_all more widely, and working on Windows would be nice for this :-)

Sebastian Ullrich (Jun 14 2024 at 13:16):

Why not just apply the workaround from above for now?

Damiano Testa (Jun 14 2024 at 18:00):

Eventually, the script could disengage completely from Mathlib, in fact.

Michael Rothgang (Jun 14 2024 at 18:12):

Sebastian Ullrich said:

Why not just apply the workaround from above for now?

That is, adding sorting to the script and applying the patch from this message to mathlib? Sounds like two mathlib PRs then (the first of which is 13777).

Michael Rothgang (Jun 14 2024 at 18:13):

By the way: why does mkAll need supportsInterpreter? Is this some obvious detail I'm not seeing?

Damiano Testa (Jun 14 2024 at 19:05):

I suspect that the supportInterpreter was cargo-culted (Yaël wrote most of the executable part of the command, so they may be able to comment on this better).

Yaël Dillies (Jun 14 2024 at 19:08):

Yeah that was not a conscious choice

Michael Rothgang (Jun 15 2024 at 09:27):

I just checked: it is in fact needed -> let me PR the patch above.

Michael Rothgang (Jun 15 2024 at 09:38):

Filed #13850 to fix mkAll and pole on Windows.
Can some Windows users (e.g. @Floris van Doorn @Utensil Song) help me test this?

  • head to the PR, check out the branch
  • run lake exe mk_all resp. lake exe pole
    Tell me if this runs without errors. Would be much appreciated!

Utensil Song (Jun 15 2024 at 09:46):

I think it's worth it to set up a Ubuntu/Windows/Mac CI for simple tools like this. I've been testing this via CI anyway.

Rida Hamadani (Jun 15 2024 at 19:59):

I've tested mathlib4#13850 and mathlib4#13854 on my windows laptop.
Both gave very similar errors:

Γ£û [5/8] Building mk_all
trace: .> LEAN_PATH=.\.\.lake/packages\batteries\.lake\build\lib;.\.\.lake/packages\Qq\.lake\build\lib;.\.\.lake/packages\aesop\.lake\build\lib;.\.\.lake/packages\proofwidgets\.lake\build\lib;.\.\.lake/packages\Cli\.lake\build\lib;.\.\.lake/packages\importGraph\.lake\build\lib;.\.\.lake\build\lib PATH c:\Users\User\.elan\toolchains\leanprover--lean4---v4.9.0-rc2\bin\lean.exe -Dpp.unicode.fun=true -DautoImplicit=false -DrelaxedAutoImplicit=false .\.\.\scripts\mk_all.lean -R .\.\.\scripts -o .\.\.lake\build\lib\mk_all.olean -i .\.\.lake\build\lib\mk_all.ilean -c .\.\.lake\build\ir\mk_all.c --json
error: .\.\.\scripts\mk_all.lean:6:0: failed to read file '.\.\.lake/packages\Cli\.lake\build\lib\Cli\Basic.olean', invalid header
error: .\.\.\scripts\mk_all.lean:17:5: unknown namespace 'Lean'
error: .\.\.\scripts\mk_all.lean:19:5: unknown namespace 'Lake'
error: .\.\.\scripts\mk_all.lean:24:18: unknown identifier 'IO'
error: .\.\.\scripts\mk_all.lean:24:18: unknown constant 'sorryAx'
error: .\.\.\scripts\mk_all.lean:26:33: expected token
error: .\.\.\scripts\mk_all.lean:35:5: unknown namespace 'IO.FS'
error: .\.\.\scripts\mk_all.lean:38:21: unknown identifier 'Parsed'
error: .\.\.\scripts\mk_all.lean:38:21: unknown constant 'sorryAx'
error: .\.\.\scripts\mk_all.lean:46:35: expected token
error: .\.\.\scripts\mk_all.lean:67:20: '`'; expected '`('
error: .\.\.\scripts\mk_all.lean:67:12: unknown identifier 'Cmd'
error: .\.\.\scripts\mk_all.lean:81:17: unknown identifier 'List'
error: .\.\.\scripts\mk_all.lean:81:17: unknown constant 'sorryAx'
error: Lean exited with code 1
Some builds logged failures:
- mk_all
error: build failed

Rida Hamadani (Jun 15 2024 at 20:01):

I would be happy to get into a call with screen sharing in order to investigate this.

Yaël Dillies (Jun 15 2024 at 20:34):

I think I've figured out how to fix this one: Delete .lake, then run the command again

Yaël Dillies (Jun 15 2024 at 20:34):

It seems that somehow lake calls the outdated executable instead of recompiling?

Rida Hamadani (Jun 15 2024 at 20:39):

Nice, that worked on my machine too!

Michael Rothgang (Jun 15 2024 at 20:44):

To summarize: the code with the PR works on your machine, which runs Windows? That is excellent news, thanks a lot for testing!

Rida Hamadani (Jun 15 2024 at 20:48):

Noo sorry for the misunderstanding!

Rida Hamadani (Jun 15 2024 at 20:48):

The code didn't work until I did Yaël's suggestion, which is to delete .lake first.

Rida Hamadani (Jun 15 2024 at 20:48):

Before doing so, I had the error that I showed above

Michael Rothgang (Jun 15 2024 at 20:53):

That I understood - what happened after deleting .lake?

Rida Hamadani (Jun 15 2024 at 20:57):

It worked. I thought you read the last message only. :sweat_smile:
But is it possible to make it so that we don't have to delete .lake?

Michael Rothgang (Jun 15 2024 at 21:01):

Good question, I don't know. Perhaps that's a one-time event which will not be too disruptive?

Rida Hamadani (Jun 15 2024 at 21:02):

Also, I've only tried mk_all, trying pole, it gave me the following error:

uncaught exception: object file '.\.\.lake\build\lib\Mathlib.olean' of module Mathlib does not exist

Michael Rothgang (Jun 15 2024 at 21:58):

Wild guess: that sounds like a need to clear .lake again

Rida Hamadani (Jun 15 2024 at 22:07):

Got the same error after deleting .lake

Utensil Song (Jun 16 2024 at 04:29):

You need lake -R build after such modification to the lakefile as demonstrated in my Windows CI.

Utensil Song (Jun 16 2024 at 04:32):

When dealing with such a delicate matter, a reproducible CI workflow is your friend and any local success could be volatile.

Utensil Song (Jun 16 2024 at 04:34):

If you have used -R, you wouldn't need to delete .lake, now you need to fetch Mathlib cache again.

Floris van Doorn (Jun 18 2024 at 08:57):

With -R these commands work withour error for me.

Floris@Dell-E MINGW64 ~/projects/mathlibf (MR-fix-mkall-windows)
$ lake exe -R mk_all
 [2/8] Built Cli.Basic
 [4/8] Built mk_all
 [5/8] Built Cli.Basic:c.o
 [6/8] Built Mathlib.Util.GetAllModules:c.o
 [7/8] Built mk_all:c.o
 [8/8] Built mk_all
Updating 'Mathlib.lean'

Floris@Dell-E MINGW64 ~/projects/mathlibf (MR-fix-mkall-windows)
$ lake exe -R pole
 [21/66] Built Batteries.Classes.BEq:c.o
 [24/66] Built Batteries.Lean.Name:c.o
 [25/66] Built Batteries.Lean.SMap:c.o
 [27/66] Built Batteries.Classes.SatisfiesM:c.o
 [28/66] Built Batteries.Data.Array.Monadic:c.o
 [29/66] Built Batteries.Data.Nat.Basic:c.o
 [30/66] Built Batteries.Data.String.Basic:c.o
 [34/66] Built Batteries.Data.AssocList:c.o
 [35/66] Built Batteries.Lean.Position:c.o
 [36/66] Built Batteries.CodeAction.Deprecated:c.o
 [37/66] Built ImportGraph.Lean.Name
 [38/66] Built LongestPole.SpeedCenterJson
 [39/66] Built Mathlib.Data.String.Defs:c.o
 [40/66] Built ImportGraph.CurrentModule
 [41/66] Built Cli.Extensions
 [42/66] Built Batteries.Lean.NameMap:c.o
 [43/66] Built ImportGraph.RequiredModules:c.o
 [44/66] Built Batteries.Data.List.Init.Lemmas:c.o
 [45/66] Built Batteries.CodeAction.Attr:c.o
 [46/66] Built Batteries.Lean.IO.Process:c.o
 [47/66] Built Batteries.Lean.Util.Path:c.o
 [48/66] Built ImportGraph.Imports:c.o
 [49/66] Built Batteries.CodeAction.Basic:c.o
 [50/66] Built Batteries.Data.HashMap.Basic:c.o
 [51/66] Built LongestPole.SpeedCenterJson:c.o
 [52/66] Built ImportGraph.Lean.Name:c.o
 [53/66] Built Mathlib.Mathport.Rename:c.o
 [54/66] Built Batteries.Tactic.Alias:c.o
 [55/66] Built Cli.Extensions:c.o
 [56/66] Built Batteries.Data.List.Basic:c.o
 [57/66] Built Cli
 [58/66] Built ImportGraph.CurrentModule:c.o
 [59/66] Built ImportGraph.Cli
 [60/66] Built Cli:c.o
 [61/66] Built ImportGraph.Cli:c.o
 [62/66] Built ImportGraph
 [63/66] Built LongestPole.Main
 [64/66] Built ImportGraph:c.o
 [65/66] Built LongestPole.Main:c.o
 [66/66] Built pole
Analyzing Mathlib at 78f57a416ef003c0b6e4d974faf97020ffd83425
http://speed.lean-fro.org says: could not find run
Try moving to an older commit?

Yaël Dillies (Jun 18 2024 at 14:19):

But what if you first run them on a different Lean version, then switch versions and run them again?

Floris van Doorn (Jun 18 2024 at 17:32):

The things I did were

  • checkout branch (from a much older Lean version)
  • lake exe cache get
  • then the displayed commands above

So with -R this just works.

Yaël Dillies (Jun 18 2024 at 17:35):

Does that mean you ran lake exe mk_all on two different Lean versions?

Floris van Doorn (Jun 18 2024 at 17:56):

Oh, I see what you mean... No I didn't do that...

Yaël Dillies (Jun 18 2024 at 17:58):

Disclaimer: I think there are actually two bugs going on in this thread, and I suspect you have found a way to fix the one I didn't report

Utensil Song (Jun 19 2024 at 01:35):

I guess .lake has the old lakefile.olean before the branch switch, so -R is required.

Kim Morrison (Jun 20 2024 at 03:01):

@Mac Malone, what are our prospects of getting rid of lakefile.olean entirely? This is still causing much suffering. :-)

Mac Malone (Jun 20 2024 at 03:02):

@Kim Morrison I have a plan on how to remove the need for -R altogether, but I have not found the time to implement it.


Last updated: May 02 2025 at 03:31 UTC