Question: how to apply getInputs() recursively?

[MatthieuStigler]

I am interested in using getInputs() recursively, so that for each function within a call, I get the output of getInputs(). The problem is that otherwise, if a call contains multiple @function, I will not know which @input or @strings belongs to it.

I see that there is a mention Note that this is not recursive. in the help file, but I was hoping someone might have a hint on how to make it recursive? The idea is that the output would be a list of getInput(), for each function?

Thanks!

Here would be two example where it is not clear to which function each input or string belong to?

library(CodeDepends)
expr <- quote(if(TRUE) {fun1(inp1)} else {fun2(input2)})
getInputs(expr)
#> An object of class "ScriptNodeInfo"
#> Slot "files":
#> character(0)
#> 
#> Slot "strings":
#> character(0)
#> 
#> Slot "libraries":
#> character(0)
#> 
#> Slot "inputs":
#> [1] "inp1"   "input2"
#> 
#> Slot "outputs":
#> character(0)
#> 
#> Slot "updates":
#> character(0)
#> 
#> Slot "functions":
#>   if    { fun1 fun2 
#>   NA   NA   NA   NA 
#> 
#> Slot "removes":
#> character(0)
#> 
#> Slot "nsevalVars":
#> character(0)
#> 
#> Slot "sideEffects":
#> character(0)
#> 
#> Slot "code":
#> if (TRUE) {
#>     fun1(inp1)
#> } else {
#>     fun2(input2)
#> }
expr <- quote(if(file.exists("file.csv")) {read.csv("file2.csv")} else {write.csv("file2.csv")})
getInputs(expr)
#> An object of class "ScriptNodeInfo"
#> Slot "files":
#> character(0)
#> 
#> Slot "strings":
#> [1] "file.csv"  "file2.csv"
#> 
#> Slot "libraries":
#> character(0)
#> 
#> Slot "inputs":
#> character(0)
#> 
#> Slot "outputs":
#> character(0)
#> 
#> Slot "updates":
#> character(0)
#> 
#> Slot "functions":
#>          if file.exists           {    read.csv   write.csv 
#>          NA          NA          NA          NA          NA 
#> 
#> Slot "removes":
#> character(0)
#> 
#> Slot "nsevalVars":
#> character(0)
#> 
#> Slot "sideEffects":
#> character(0)
#> 
#> Slot "code":
#> if (file.exists("file.csv")) {
#>     read.csv("file2.csv")
#> } else {
#>     write.csv("file2.csv")
#> }

^{Created on 2025-02-18 with reprex v2.1.1}

[duncantl]

Hi Matthieu
If you are willing to install and use the CodeAnalysis package, the code here gets you places. Hope that helps.
I have to run to a meeting but wanted to note that it might be good for us to back up and figure out exactly what we want to have happen if analyzing a script. Is it just the if() statements we care about or what about while(), for().
This is because in R everything is a function call.

[MatthieuStigler]

Dear Duncan

Thanks a lot for your prompt answer! I like your answer a lot, though indeed the restriction of having to load two packages not on CRAN is a bit constraining?

But yes, backing up might be helpful: I am just trying to represent my input/output script dependencies, i.e. script_1 inputs file_1 and output file_2, while script_2 inputs file_2 and creates figures_1. So I am trying in a first stage to:

1. screen all functions, and keep only those such as read.csv, write.csv, sf::st_read, etc
2. figure out their input (and I am ready to only consider "path/file.csv" objects, and not assignements, i.e. I am considering write.csv("file.csv") but discarding a <- "file.csv"; write.csv(a)
3. get a clean df of script_name; function; path

This is pretty much working so far, except for nested functions, where a if() call would introduce some ambiguity on the which input/path/string belongs to which function.

Do you think there is a solution without external dependencies? I had tried this, but seems not very elegant (and not truly recursive either).

Thanks so much!

PS: we've met a few times when I was doing my PhD at Davis :-)

library(CodeDepends)
expr1 <- quote(if(TRUE) {fun1(inp1)} else {fun2(input2)})
expr2 <- quote(if(file.exists("file.csv")) {read.csv("file2.csv")} else {write.csv("file2.csv")})
getInputs_multi <- function(expr, ...){
  out <- CodeDepends::getInputs(expr)
  if(length(out@functions)>1){
    sub_calls <- as.list(expr)
    fo_names <- names(out@functions)
    sub_calls_really_call <- sapply(sub_calls, is.call)
    if(sum(sub_calls_really_call)>0) sub_calls <- sub_calls[sub_calls_really_call]
    out <- lapply(sub_calls, \(x) CodeDepends::getInputs(x, ...))
    names(out) <- fo_names[sub_calls_really_call]
  }
  out
}
getInputs_multi(expr1)|> lapply(slot, "inputs")
#> $fun1
#> [1] "inp1"
#> 
#> $fun2
#> [1] "input2"
getInputs_multi(expr2)|> lapply(slot, "strings")
#> $file.exists
#> [1] "file.csv"
#> 
#> $`{`
#> [1] "file2.csv"
#> 
#> $read.csv
#> [1] "file2.csv"

^{Created on 2025-02-19 with reprex v2.1.1}

[MatthieuStigler]

Dear Duncan

I was wondering whether you would have a chance to share some guidance on how I could use getInputs() recursively without using external packages that are not on CRAN? Thanks a lot!

[duncantl]

Hi Matthieu.
Firstly, nice to "see" you again. Thanks for giving me the context of your PhD at Davis. I do recall when I got to see your picture. I hope you are doing well.

If a package was on another repository, not CRAN, would that be okay? And does it have to be available in binary form or installable from source via install.packages( "CodeAnalysis", repos = "https://....")?
Is this for your own use, or others using your code?

For various reasons, I stay clear of CRAN. Gabe (Becker) put CodeDepends on CRAN. The small change to fix issue #52 may not make it to CRAN.

However, I could put CodeDepends and CodeAnalysis on a different repository or one could use the devtools package (or similar) to install directly from github. One can add multiple repositories to options(repos = ...) and it could then find the package "transparently."

[MatthieuStigler]

Hi Duncan! Glad you remember me, from the good old time back in Davis :-)

Actually that is not such a big deal if the packages are on github (wish I would meet you live to hear your reasons to avoid CRAN), chances that I push this own code into a CRAN package are fairly low.

I have been struggling with this idea of a recursive function, turns out it is way more complicated than I thought. Thanks to your help, I got one step further, but am now encountering a new difficulty/unexpected behavior, see below.

But maybe I should step back at this point: my goal is actually to obtain the (string) input of any functions belonging to some group (say read.csv, write.csv, etc), not to any function in general. Maybe starting from this, there is an easier way to address this?

Anyway, below is the current problem I am facing. Thank you so much for your help!

q <- quote(cc_dt %>% mutate_at(cc_dt_covs, as.numeric))

## works
res <- CodeDepends::getInputs(q)

## doesn't work
source("https://raw.githubusercontent.com/duncantl/CodeAnalysis/refs/heads/master/explorations/Matthieu.R")
lapply(q, CodeDepends::getInputs)
#> Error in e[[i]]: subscript out of bounds
foo(q)
#> Error in e[[i]]: subscript out of bounds

^{Created on 2025-03-10 with reprex v2.1.1}

[duncantl]

Hi Matthieu
Thanks for the answer about CRAN/github/others using your code. That helps understand your constraints.

What you are doing is very similar to one of the applications we had envisioned for the CodeAnalysis package and explored/partially implemented in some examples/case-studies, at least.

CodeDepends could be "tortured" into doing the analysis you want. However, it is designed for scripts and top-level commands. I think you are working uphill to make it do what you want.

I think your description involves

• searching for all function calls
• filtering based on what function is being called and
• examining the arguments.

That's what CodeAnalysis does well. That said, CodeAnalysis depends on several packages.
So I created a separate simplified version of findCallsTo() to find all function calls. But then I started to tweak that and .... Now there is an exploratory package - FindCalls - that I think will be simpler to install (from github source - no dependencies other than codetools that comes with R) and is actually now pretty general for your purposes.

It is a work in progress but has an example that works through various aspects of what you are doing. Unfortunately, the project it analyzes is not public so I can't share. Perhaps ask Matt Espe for access if changing the dir = to one of your own projects causes many problems.

I am thinking about whether to either move some of the functionality from CodeAnalysis to this package or reduce the required dependencies so CodeAnalysis can be installed much more easily. If I decide to further this FindCalls package, I'll add documentation.

[MatthieuStigler]

Hi Duncan

Wow, this gets me very close to where I want I feel. I need a bit more time to check in detail how it fits to my case, but thanks a lot for sharing that code, much appreciated!

PS: probably a missing parenthesis in ReadFunArgName = rep(NA, length(ReadFuns) line 87?

[duncantl]

Thanks. Fixed the missing ) and added some comments to explain some of the other aspects one would ideally deal with.