JyNI was accepted for GSoC once again!

2017 GSoC project page
2015 GSoC project blog

Huge thanks to everyone who made this possible!
Most notably to Jim Baker for mentoring the entire proposal and the project itself.
Further thanks go to Frank Wierzbicki and Darjus Loktevic for mentoring this brand new 2017 JyNI GSoC project.

Jython: Develop Windows version of JyNI and ideally support all extensions currently workable on POSIX (e.g. ctypes, NumPy)

Abstract

JyNI is a compatibility layer with the goal to enable Jython to use native CPython extensions like NumPy or SciPy. So far it was developed on Linux and some additional effort was spent to let the core part work properly on OSX. In principle the current version of JyNI is workable on POSIX platforms. I propose to create a proper Windows version of JyNI. A fully accurate implementation of this proposal includes to establish workability of all extensions and features that are currently supported on POSIX platforms.

Detailed Description

Windows uses a different loading mechanism for dynamic libraries than POSIX (https://en.wikipedia.org/wiki/Dynamic_loading#Summary). Essentially the adjustment for a Windows version of JyNI can be ported from CPython. However my earlier efforts to support OSX showed that a new target platform likely yields subtle issues (especially regarding memory management) all through JyNI code. Also note that previously no care was taken to produce Windows-compliant code and there are various sections in JyNI that need adjustments due to this fact. In order to support a CPython compatible ABI on Windows, I will have to build JyNI using (a certain, non-current version of) MS Visual Studio, which does not fully support C99 and some parts of JyNI will need adjustment due to this fact. I will eventually try to use mingwpy (https://github.com/mingwpy/mingwpy) as an alternative toolchain, but that project is still experimental.
After basic porting is done, I will step by step work through JyNI's Demo extension and test suite to ensure proper workability of all parts. Finally I will work through various examples and extensions that are currently known to be workable with JyNI on POSIX in order to finally achieve a full-featured Windows-version of JyNI.


Comments

Post a Comment

Popular posts from this blog

Final report: ctypes and NumPy work with JyNI on Windows

After the last report, where I managed to run DemoExtension with JyNI on Windows, I investigated workability of ctypes. Linking msvcrt The first thing I discovered was that it didn't work: Exception in thread "main" java.lang.UnsatisfiedLinkError: D:\workspace\linux\JyNI\build\JyNI.dll\python27.dll: Can't find dependent libraries And further – that DemoExtension also was not working when built using setup.py rather than by my makefile . That was a drawback, but remember that DemoExtension was working fine with CPython, even if built with the makefile. After carefully studying the difference between the build commands I found out that setup.py would add the /MD flag . Beneath several other options, e.g. /nologo /Ox /W3 /GS- /DNDEBUG, /MD was the one that made the difference. It tells MSVC to link msvcrt.dll , precisely speaking msvcr90.dll for Python 2.7. I guess the DemoExtension build without explicit /MD worked, because it still linked libcmt.lib. However
Read more

The motto of my last few days: "error LNK2019: unresolved external symbol"

Work on linking/error LNK2019: unresolved external symbol The last two weeks I spent on fixing unresolved external symbols. While it was straight forward with GCC and clang to ignore unresolved symbols (i.e. implicit definitions in GCC or by passing "-undefined dynamic_lookup" to clang) at compile time, there seems to be no such option with MSVC. Anyway, it's not a bad idea to fix these symbols as they can become runtime errors at some point, so tried to get this done. Here is an excerpt from the list of failing symbols: dl_nt.obj : error LNK2019: unresolved external symbol __imp_LoadStringA referenced in function DllMain unicodeobject.obj : error LNK2019: unresolved external symbol __imp_CharPrevA referenced in function is_dbcs_lead_byte stringobject.obj : error LNK2019: unresolved external symbol stringlib_partition referenced in function string_partition stringobject.obj : error LNK2019: unresolved external symbol stringlib_rpartition referenced in function stri
Read more

Lessons learned with Visual Studio, C89 and the preprocessor

Another two weeks of GSoC are done. Many lessons learned, many problems hit, most of them solved. Overview The first three days I spent with getting the original setup for building CPython 2.7 on Windows in line. After that I concentrated on aligning this with JyNI's usual build process. It quickly turned out that already JyNI's DemoExtension was not C89 compliant. After fixing this , I got the demo extension working on original CPython 2.7, which asserts that the ABI is correct. With this in line, I focused on JyNI itself and checked for an automatic C99 to C89 approach . It turned out that this approach wouldn't be workable without additional efforts , so I started to manually convert the JyNI codebase to C89. During this work I suffered from frequent mouse freezes on Windows 10. While these always occurred during working with Eclipse I cannot tell whether it was Eclipse related. Also for other reasons I preferred to return to my Linux environment and checked
Read more