GSoC 2017 progress on JyNI

After fixing the remaining symbol errors MSVC successfully created JyNI.dll for the very first time – what a historical moment! Unfortunately the euphoria didn't last long, because when loading DemoExtension , it directly crashed. After some investigation I found out that JyNI.dll was successfully loaded by the JVM and I also verified that code in the dll was actually accessed and properly run. Even the initial call into DemoExtension.pyd worked. However, the call to Py_InitModule4 crashed: It seemed like symbols from JyNI.dll were not exported for access by DemoExtension.pyd. Strangely, the error message wasn't like one would expect for this. It rather looked like DemoExtension.pyd was calling into CPython's python27.dll, i.e. the one installed on the system. The definite proof of concept After some reading in MSDN docs I learned that the names of dll dependencies are hard coded at compile time, which means that DemoExtension.pyd will insist on loading python27.d...

Work on linking/error LNK2019: unresolved external symbol The last two weeks I spent on fixing unresolved external symbols. While it was straight forward with GCC and clang to ignore unresolved symbols (i.e. implicit definitions in GCC or by passing "-undefined dynamic_lookup" to clang) at compile time, there seems to be no such option with MSVC. Anyway, it's not a bad idea to fix these symbols as they can become runtime errors at some point, so tried to get this done. Here is an excerpt from the list of failing symbols: dl_nt.obj : error LNK2019: unresolved external symbol __imp_LoadStringA referenced in function DllMain unicodeobject.obj : error LNK2019: unresolved external symbol __imp_CharPrevA referenced in function is_dbcs_lead_byte stringobject.obj : error LNK2019: unresolved external symbol stringlib_partition referenced in function string_partition stringobject.obj : error LNK2019: unresolved external symbol stringlib_rpartition referenced in function stri...

During the last two weeks I almost completed the C99 to C89 conversion process . This was a lot of work, but mostly routine. The only mentionable issue was to use PyAPI_FUNC correctly . Especially it wasn't obvious to me that building CPython on Windows requires the Py_BUILD_CORE preprocessor definition . This stuff is usually automated by configure script or setup.py. For JyNI I prefer to have a transparent toolchain configuration, at least at this stage. As of this writing, the only pieces not yet ported are dynload_shlib and JyNI-loader . Both involve switching to dll loading mechanism and thus more thought. I.e. bridging POSIX and Windows dynamic loading machanisms in suitable crossplatform manner. Getting this right and doing the linking phase is the next step on the agenda.

Another two weeks of GSoC are done. Many lessons learned, many problems hit, most of them solved. Overview The first three days I spent with getting the original setup for building CPython 2.7 on Windows in line. After that I concentrated on aligning this with JyNI's usual build process. It quickly turned out that already JyNI's DemoExtension was not C89 compliant. After fixing this , I got the demo extension working on original CPython 2.7, which asserts that the ABI is correct. With this in line, I focused on JyNI itself and checked for an automatic C99 to C89 approach . It turned out that this approach wouldn't be workable without additional efforts , so I started to manually convert the JyNI codebase to C89. During this work I suffered from frequent mouse freezes on Windows 10. While these always occurred during working with Eclipse I cannot tell whether it was Eclipse related. Also for other reasons I preferred to return to my Linux environment and checked...

So, the first two weeks of GSoC are over now and I used the time to finalize JyNI 2.7-alpha.4 release. The project achieved an initial NumPy support in late 2016, but that milestone was not releasable until now, because it requires Jython 2.7.1, which suffered an unfortunate delay due to several release-blocking issues. Recently, Jython 2.7.1RC2 was released , which I regarded as sufficient base for a new JyNI release. Even more important, fixing up loose ends for a release was the ideal work to get back into project detail after more than half a year. First Week Working on the release, I soon recognized some memory flaws on a 32 bit system, which I could luckily solve as of da5fc5c (debugging this stuff on my old 32 bit desktop machine is a pain). In earlier work on JyNI I had added some control for how specific extensions perceive platform detection via os.name and sys.platform on Jython using techniques one might call "Python black magic" (see follow up report of J...

JyNI was accepted for GSoC once again! 2017 GSoC project page 2015 GSoC project blog Huge thanks to everyone who made this possible! Most notably to Jim Baker for mentoring the entire proposal and the project itself. Further thanks go to Frank Wierzbicki and Darjus Loktevic for mentoring this brand new 2017 JyNI GSoC project. Jython: Develop Windows version of JyNI and ideally support all extensions currently workable on POSIX (e.g. ctypes, NumPy) Abstract JyNI is a compatibility layer with the goal to enable Jython to use native CPython extensions like NumPy or SciPy. So far it was developed on Linux and some additional effort was spent to let the core part work properly on OSX. In principle the current version of JyNI is workable on POSIX platforms. I propose to create a proper Windows version of JyNI. A fully accurate implementation of this proposal includes to establish workability of all extensions and features that are currently supported on POSIX platforms. ...