Sync with upstream v4.0.29 #66
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…nio#4612) Just some fixed MAX_BATCH_SIZE until we come up with something better. Fixes: joernio#4611
…ariables (joernio#4617) * Add reproducing test * Add tests for new lambdas * Never add Unknown node for captured variable * Add todos to comments about lambda parameter capturing
* [python] prevent overtainting dictLiterals * use allAssignmentTypes in isAssignment * use methodFullNameExact instead of nameExact in isAssignment
As pointed out in joernio#4240, combining this nested immutable map-like structure has a quadratic performance, and the more performant strategy would be to use nested data-structures to merge. For now, I've decided not to opt for a builder pattern, but rather keep the underlying structure mutable, and accessor methods return immutable structures.
Added a recursive aspect to the call type recovery if the receiver of the call is possibly recoverable (and it's receiver if that's a call and so on)
- log cpg creation for each file in DEBUG only instead of INFO - log the astgen path only once with INFO - removed logging that astgen runs now in the given input folder
- Fix last error token position if error token itself is semicolon, newline or EOF. - Also log the parser exception message.
Moved parser logging of memory clearing from INFO to DEBUG
A common pattern in JavaScript is something like
```javascript
var foo = {};
foo.bar = {};
foo.bar.someFunc = function someFunc() {};
foo.bar.someFunc();
```
This PR adds a post-processing pass to find instances where the definition and the call live in the same file and to link them. This "common file" limitation aims to reduce false linking.
also to minify the diff for the flatgraph port
Allow the frontend to continue analysis if the type stubs for built-ins is missing.
The `new` operator can prefix an ordinary call, e.g.
```javascript
var Print = function(str) {
console.log(str)
}
new Print("Hello")
```
The type recovery pass only considers object instantiations from type declarations on `new` calls, but this change allows it to consider functions too.
Other changes include adding constants for `:program` and `<operator>.new`
Added implicit self receiver to simple calls.
In ruby as for all other dynamic language there is no vtable and thus only a single entry in the binding tables with name=="" and signature=="".
This change moves the type/method identifier references for entities exportable from the script to prefix the respective entity at the definition.
this caused me some headaches as I had an old jar lying around there, likely from a bisect session...
This change moves the type/method identifier references for entities exportable from the script to prefix the respective entity at the definition.
ICPPASTExpression.getEvaluation may fail throwing an unrecoverable exception in case of unresolved includes etc.
* Added `Member` nodes for each method, to relate to their respective bound `TypeDecl` nodes. * Added a type decl for "fake methods" that include the methods and classes exported by the file
To simplify the call graph, methods resolved to the `Kernel` class are dispatched as static. Additionally, replaced `__builtin` with `Kernel` to map more closely to Ruby's internals and implemented `MemberAccessCommandContext`.
* [ruby] Added handling for String interpolated array literals * [ruby] Added type check in tests * cleanup * [ruby] Fixed parser tests with new parser test framework * [ruby] removed if check on element content
* [ruby] handling added for command literal. Modelled as exec call * [ruby] fixed failing parser test * [ruby] Removed print * [ruby] Added expanded command literal to new parser tests
Using the old SerializedCpg API hat no effect anymore.
* [ruby2cpg] Fix ImplicitRequirePass. - Fix method lookup via AST edges. The approach of looking up module methods via is not possible at the execution time of the pass because AST is not yet linked. I replaced this buy a fullname based regex lookup. We should likely just change the execution time to after AST linking. - The other changes are just for better readability and debugability. * Address review remarks. * Fix field access lookup. We are only interested on those field accesses which operator on "self" since those describe the exported/imported objects.
* [ruby] Fixed bracket assignments * [ruby] Add parser test for BracketAssignment
…oernio#4680)" This reverts commit 195f015.
This reverts commit dd53acd.
joern-cli/frontends/csharpsrc2cpg/src/main/scala/io/joern/csharpsrc2cpg/astcreation/AstForStatementsCreator.scala
joern-cli/frontends/jssrc2cpg/src/main/scala/io/joern/jssrc2cpg/Main.scala joern-cli/frontends/jssrc2cpg/src/main/scala/io/joern/jssrc2cpg/utils/AstGenRunner.scala
joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/passes/base/ContainsEdgePass.scala joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/passes/base/FileCreationPass.scala joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/passes/base/MethodStubCreator.scala joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/passes/base/NamespaceCreator.scala joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/passes/base/ParameterIndexCompatPass.scala joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/passes/base/TypeEvalPass.scala joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/passes/base/TypeRefPass.scala joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/passes/callgraph/DynamicCallLinker.scala joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/passes/callgraph/MethodRefLinker.scala joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/passes/controlflow/CfgCreationPass.scala joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/passes/controlflow/cfgdominator/CfgDominatorPass.scala joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/passes/typerelations/AliasLinkerPass.scala joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/passes/typerelations/FieldAccessLinkerPass.scala joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/passes/typerelations/TypeHierarchyPass.scala joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/utils/LinkingUtil.scala
Joern4 branch Sync
We trigger the post-processing directly if a cpg is loaded on startup (e.g. `./joern cpg.bin`). In that case, the typeRecoveryConfig has not been initialized. There's a reason why in Scala you have to explicitly make an effort to start with `null`, because more often than not you'll be missing out some random edge case... lazy val is the better alternative fixes joernio#4999 (again...)
fixup initialisation order of cpg generators (joernio#5137)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
val cpgVersion = "0.1.8+8-a9ecde94+20240806-2347"needs to be upgraded to the released version