Ubuntu: Julia dependencies fail with `patter::julia_connect`, succeed if installed directly

[mhpob]

HDF5_jll, NetCDF_jll, GDAL_jll, GDAL, ArchGDAL, GeoArrays, and Rasters all fail to build if using patter::julia_connect() within R. Similarly, building RCall fails from within patter/JuliaCall.

All build without issue if adding from within Julia.

Installing RCall fails as it can't find libquadmath.so.0, which is within the gcc system library and so is fixed by installing gcc.

On the ArchGDAL/GeoArrays/Raster front, it may to come from not having OpenSSL v3.3.0 installed on the system, but Ubuntu Noble only has OpenSSL v 3.0.13

... Handling dependencies... 
Error: Error happens in Julia.
InitError: could not load library "/root/.julia/artifacts/a696e1e3e6b8d3c2c4b67609357ebc5c35eabde1/lib/libssl.so"
/usr/lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.3.0' not found (required by /root/.julia/artifacts/a696e1e3e6b8d3c2c4b67609357ebc5c35eabde1/lib/libssl.so)

I'm unfamiliar with Julia, but as Julia installs packages without issue while patter goes looking for system libraries perhaps Julia, itself, is installing precompiled binaries while RCall/patter is looking to build them, themselves?

[mhpob]

Patter install also breaks when using patter::julia_connect() due to OpenSSL but installs fine in Julia.

julia_connect also seems "sticky" between sessions, showing the same error that stopped it before without re-running.

[mhpob]

I ran Patter.jl through Julia and installation/dependency resolution worked fine. Still issues when using patter::julia_connect():

> julia -e 'using Pkg; Pkg.add(url = "https://github.com/edwardlavender/Patter.jl")'
#no issues

> patter::julia_connect()
`patter::julia_connect()` called @ 2025-04-30 18:17:11... 
... Running `Julia` setup via `JuliaCall::julia_setup()`... 
Julia version 1.11.5 at location /usr/local/julia/bin will be used.
Loading setup script for JuliaCall...
Finish loading setup script for JuliaCall.
... Validating Julia installation... 
Using Julia's global environment.
... Handling dependencies... 
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `~/.julia/environments/v1.11/Project.toml`
  [c9ce4bd3] + ArchGDAL v0.10.8
  No Changes to `~/.julia/environments/v1.11/Manifest.toml`
   Resolving package versions...
    Updating `~/.julia/environments/v1.11/Project.toml`
  [a93c6f00] + DataFrames v1.7.0
  No Changes to `~/.julia/environments/v1.11/Manifest.toml`
   Resolving package versions...
    Updating `~/.julia/environments/v1.11/Project.toml`
  [31c24e10] + Distributions v0.25.119
  No Changes to `~/.julia/environments/v1.11/Manifest.toml`
   Resolving package versions...
    Updating `~/.julia/environments/v1.11/Project.toml`
  [2fb1d81b] + GeoArrays v0.9.4
  No Changes to `~/.julia/environments/v1.11/Manifest.toml`
   Resolving package versions...
    Updating `~/.julia/environments/v1.11/Project.toml`
  [033835bb] + JLD2 v0.5.13
  No Changes to `~/.julia/environments/v1.11/Manifest.toml`
   Resolving package versions...
    Updating `~/.julia/environments/v1.11/Project.toml`
  [a3a2b9e3] + Rasters v0.14.4
  No Changes to `~/.julia/environments/v1.11/Manifest.toml`
Error: Error happens in Julia.
InitError: could not load library "/root/.julia/artifacts/a696e1e3e6b8d3c2c4b67609357ebc5c35eabde1/lib/libssl.so"
/usr/lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.3.0' not found (required by /root/.julia/artifacts/a696e1e3e6b8d3c2c4b67609357ebc5c35eabde1/lib/libssl.so)
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:120
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:119
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/GfYNv/src/products/library_generators.jl:63 [inlined]
  [4] __init__()
    @ OpenSSL_jll ~/.julia/packages/OpenSSL_jll/WaTt1/src/wrappers/x86_64-linux-gnu.jl:16
  [5] run_module_init(mod::Module, i::Int64)
    @ Base ./loading.jl:1378
  [6] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1366
  [7] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vecto
`patter::julia_connect()` call ended @ 2025-04-30 18:18:40 (duration: ~1.48 min(s)). 

[mhpob]

Running julia_connect, letting it error, then running it again with .socket=TRUE "fixes" it.

> patter::julia_connect()
`patter::julia_connect()` called @ 2025-04-30 18:36:24... 
... Running `Julia` setup via `JuliaCall::julia_setup()`... 
Julia version 1.11.5 at location /usr/local/julia/bin will be used.
Loading setup script for JuliaCall...
Finish loading setup script for JuliaCall.
... Validating Julia installation... 
Using Julia's global environment.
... Handling dependencies... 
Error: Error happens in Julia.
InitError: could not load library "/root/.julia/artifacts/a696e1e3e6b8d3c2c4b67609357ebc5c35eabde1/lib/libssl.so"
/usr/lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.3.0' not found (required by /root/.julia/artifacts/a696e1e3e6b8d3c2c4b67609357ebc5c35eabde1/lib/libssl.so)
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:120
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:119
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/GfYNv/src/products/library_generators.jl:63 [inlined]
  [4] __init__()
    @ OpenSSL_jll ~/.julia/packages/OpenSSL_jll/WaTt1/src/wrappers/x86_64-linux-gnu.jl:16
  [5] run_module_init(mod::Module, i::Int64)
    @ Base ./loading.jl:1378
  [6] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1366
  [7] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vecto
`patter::julia_connect()` call ended @ 2025-04-30 18:36:33 (duration: ~9 sec(s)). 

> patter::julia_validate()
Error: Error happens in Julia.
UndefVarError: `Patter` not defined in `Main`
Suggestion: check for spelling errors or missing imports.
Stacktrace:
 [1] top-level scope
   @ none:1
 [2] eval
   @ ./boot.jl:430 [inlined]
 [3] eval_string(x::String)
   @ Main.JuliaCall /usr/lib/R/site-library/JuliaCall/julia/setup.jl:203
 [4] docall(call1::Ptr{Nothing})
   @ Main.JuliaCall /usr/lib/R/site-library/JuliaCall/julia/setup.jl:176

> patter::julia_connect(.socket = TRUE)
`patter::julia_connect()` called @ 2025-04-30 18:36:44... 
... Running `Julia` setup via `JuliaCall::julia_setup()`... 
... Validating Julia installation... 
Using Julia's global environment.
... Handling dependencies... 
... `Julia` set up with 8 thread(s). 
`patter::julia_connect()` call ended @ 2025-04-30 18:36:46 (duration: ~2 sec(s)). 

> patter::julia_validate()

[edwardlavender]

Thanks for the issue!

What version of Ubuntu and R are you using? I can see if I can reproduce this issue. For now, it sounds like you have got things more-or-less working.

Some comments:

• julia_connect() is basically just a wrapper for JuliaCall::julia_setup()
• julia_setup() installs RCall.jl and some other packages required for R/Julia communication
• So your RCall.jl issue relates directly to JuliaCall -- its good to know that gcc solved this for you
• julia_connect() then installs Patter.jl, plus Rasters.jl and the other packages you see above. Those packages are then precompiled every time you connect to Julia (I think this explains your 'sticky' comment).

Your code

julia -e 'using Pkg; Pkg.add(url = "https://github.com/edwardlavender/Patter.jl")'

installs Patter.jl but not the other Julia dependencies used by patter (e.g., Rasters.jl). You can check what you have installed after fiddling around with using Pkg; Pkg.status(). You can install the remaining patter dependencies in Julia with:

using Pkg
Pkg.add("ArchGDAL")
Pkg.add("DataFrames")
Pkg.add("Distributions")
Pkg.add("GeoArrays")
Pkg.add("JLD2")
Pkg.add("Rasters")
Pkg.add("Random")

• On Linux, you can have some trouble when you're using R and Julia packages that rely on system dependencies. R and Julia can use different versions of these dependences and this can cause conflicts on linux. I think this is what is happening on your system.
• I have noticed this with geospatial packages (sf, terra) and julia_connect() warns if those are loaded on linux, but it sounds like there may be other conflicts on your system.
• Running julia_connect() for the first time within a completely blank R session may solve those issues for you. Note that if you're using RStudio, it may load packages 'behind the scenes' if you have autocomplete and other settings turned on, so you need to be careful (see On Linux, on RStudio, julia_connect() can crash  #16).
• You can get R and Julia to use the same versions of system libraries, but this is a system-specific challenge (see here).
• In general, on Linux it can be easier to install Patter.jl and other packages directly in Julia, and get everything working as you have done, and then all julia_connect() has to do is set up the Julia session and precompile the packages (the above considerations about system libraries still apply though). It is not uncommon to get precompilation issues that can be safely ignored if julia_connect() successfully completes.
• You still need to be careful on linux, unless you specially configure your system, that you don't use R and Julia packages that rely on different versions of the same system dependencies. For example, if you successfully connect to a Julia session, you can't then use functions that require terra in the same R session. Core functions like patter::set_map() enable you to specify a file path to a raster, rather than a SpatRaster, so that all the geospatial stuff happens in Julia. See the function documentation for details.
• Btw, there are also specific issues with JuliaCall/Ubuntu 24.04 to be aware of (see here).
• The other thing to say is that the Julia package ecosystem is less stable than the R ecosystem. That is, sometimes you get issues like the above because someone somewhere has updated a Julia package and this update hasn't fed through to all the other packages. Waiting a few days and trying again usually solves that issue. The geospatial/GDAL-reliant packages are the usual troublemakers.

I am thinking about other ways of R/Julia integration that may make use on linux easier so will keep this issue open in case others report similar experiences.

Patter.jl can also be used directly, but it focuses on model-based inference routines & doesn't contain the data assembly/processing/mapping routines that are in patter. But setting up everything in R and then running Patter.jl via a system call is another way to go.

[edwardlavender]

It may be helpful to browse the JuliaCall issues. There are some similar reports of trouble with openssl etc.

[mhpob]

Running this a few more times in different ways, it often seems like something is happening in Julia, itself, rather than anything dealing with dependencies or their method of installation. Running patter::julia_connect(.socket = T) after the first julia_connect() error then works.

What version of Ubuntu and R are you using?

I was using Ubuntu 24.04 and R 4.5.0 in a Docker container.

I've created a container with Julia, r2u, and R::patter installed, but no Julia packages/dependencies: https://github.com/mhpob/patter_containers/blob/main/Dockerfile-test

docker pull ghcr.io/mhpob/patter:test

Running docker --rm -it patter:test bash will let you tool around if you're unfamiliar with Docker.

I also made one like the above, but with patter.jl/dependencies installed:
https://github.com/mhpob/patter_container/blob/main/Dockerfile.

docker pull ghcr.io/mhpob/patter:ubuntu

You can still recreate the julia_connect(); julia_connect(.socket = TRUE) issue by running docker --rm -it patter:ubuntu R and then patter::julia_connect() in the connected session.

[edwardlavender]

Thanks for the report & the dockerfiles! I will look into this (but probably not as a matter of urgency as it sounds like you have got things working). It is not surprising on Linux to have some configuration trouble... It may be that moving away from a C linkage between R and Julia (as implemented by JuliaCall) is the way to go, but it will be good to reflect on this and see how this issue evolves and affects other users. See also this post.