Add Rust Support and Tree-Sitter Integration for JavaScript for now...

[Harsh-Kachhaway]

• This pull request adds initial Rust support and integrates the Tree-sitter library for JavaScript parsing to improve accuracy and enable future multi-language features.

• Rust support includes a basic parsing pipeline for handling core structures like functions and modules, with scope for further enhancements.

• The Tree-sitter library has been used for JavaScript parsing to improve syntax handling, error tolerance, and reliability for complex code.

• Tree-sitter is currently used within the implementation without being added as a formal project dependency.

• Testing confirms correct JavaScript parsing using Tree-sitter and basic Rust file handling with no regressions.

• These changes provide a scalable foundation for extending language support and advanced code analysis features.

[glato]

@Harsh-Kachhaway Thanks for your PR! Just checkout out your PR locally on

% python --version                                    
Python 3.12.9

but it seems that the javascript and rust unit tests are not running (debugged quickly and seems that import dependencies are not extracted with the unit test data?), ... which python version are you using for running the tests?

[Harsh-Kachhaway]

Thanks for checking it out!

I’m currently running the tests on Python 3.10 . I haven’t tested extensively on Python 3.12 yet, so that might be contributing to the issue.

Regarding the failing JavaScript and Rust tests — you’re right. The issue seems to be that Tree-sitter language dependencies (grammars) are not being properly loaded/extracted during test execution, which causes those parsers to fail.

I’ll look into:

• Ensuring the required Tree-sitter grammars are properly initialized in the test setup
• Verifying compatibility with Python 3.12
• Possibly adding missing dependency setup in the test configuration

I’ll push a fix shortly..

[Harsh-Kachhaway]

Updates & Fixes

Thanks for the review, and sorry for the delay, I was caught up with some other work.
I’ve gone through the issues and made a few updates.

• I added tree-sitter, tree-sitter-rust, and tree-sitter-javascript to requirements.txt so Rust and JavaScript parsing works properly in both local setups and CI.
• also fixed the path handling in abstractparser.py by using .as_posix(), which resolves the Windows-related test failures.
• Cleaned up the repository a bit as well by updating the .gitignore to ignore cache files.
• I ran the full test suite locally on Python 3.10 and everything is passing (25/25). For now, tree-sitter support is added for Rust and JavaScript.

As a small note, I’ll be working on extending tree-sitter support to the remaining languages as well to make things more complete.

[Harsh-Kachhaway]

Just checked .... tests pass on Python 3.10 and 3.11, but fail on Python 3.12+ due to the pkg_resources dependency.

[glato]

@Harsh-Kachhaway thanks a lot for the effort, will check this out tomorrow. In the meantime: maybe wait a couple of days, I'm preparing a bigger release which is a complete rewrite of this tool in nodes/react. This might make it easier to add/adapt further languages (it already includes rust support) and there will be more advantages and features waiting 🥁. Stay tuned 👀.

[Harsh-Kachhaway]

Thanks for the update — that sounds really exciting
I’ll wait for the new release. I’d love to contribute to the new version. I’m also interested in building something similar for personal use, so happy to collaborate and help wherever I can

[txbm]

I was literally about to rewrite it myself, glad I saw this. When's the release?

[glato]

@txbm @Harsh-Kachhaway The rewrite is about 95% done, now it's "only" bugfixing and testing. Looks good so far, plenty of performance optimizations, features. Shouldn't be too long now. Will keep you up to date, stay tuned.