The first two weeks of GSoC, or: Proud to release JyNI 2.7-alpha.4, featuring NumPy support.

So, the first two weeks of GSoC are over now and I used the time to finalize JyNI 2.7-alpha.4 release. The project achieved an initial NumPy support in late 2016, but that milestone was not releasable until now, because it requires Jython 2.7.1, which suffered an unfortunate delay due to several release-blocking issues.
Recently, Jython 2.7.1RC2 was released, which I regarded as sufficient base for a new JyNI release. Even more important, fixing up loose ends for a release was the ideal work to get back into project detail after more than half a year.

First Week

Working on the release, I soon recognized some memory flaws on a 32 bit system, which I could luckily solve as of da5fc5c (debugging this stuff on my old 32 bit desktop machine is a pain).
In earlier work on JyNI I had added some control for how specific extensions perceive platform detection via and sys.platform on Jython using techniques one might call "Python black magic" (see follow up report of JyNI GSoC 2015). By now these had made their way into Jython itself, so it was kind of a cleanup task to remove this code from JyNI and switch to the solution included in Jython.

Second Week

The major challenge was yet to come. By the middle of last week I discovered that running PyOpenGL for a while (e.g. the gear-demo) could deadlock JyNI's GC thread. While this didn't show up directly, from that moment on the program would silently start to leak memory until it would crash at some point. This turned out as a deeply involved bug in JyNI's GC support that I really liked to get fixed for the release and before I would move my focus to Windows support.

After some days of investigation it turned out that the behavior was partly due to aspects regarding resurrected objects I didn't observe until now (maybe it was introduced with Java 8): If an object was made weakly reachable again via its finalizer (kind of a "weak" resurrection), it usually took another Java GC cycle to collect it again. Now such objects were suddenly collected again within the same Java GC-cycle.
Other factors also contributed to the issue and it took me until Tuesday to understand and fix all of it. Now JyNI's GC code is working more robustly and I finally added a convenient way to assert that JyNI's GC thread is not deadlocked again (shouldn't happen any more)

The Release

After a bunch of final sanity checks I am proud to release JyNI 2.7-alpha.4, featuring initial NumPy (>= 1.12) support.

What's next

My plans for tomorrow are to setup my Windows development environment. The first thing I will check out is to build JyNI's DemoExtension for CPython 2.7 on Windows. I heard a lot of rumors that it is not easy to get the ABI right. It is said to require some certain older version of MS-VS. Keep fingers crossed...


Popular posts from this blog

Final report: ctypes and NumPy work with JyNI on Windows

Lessons learned with Visual Studio, C89 and the preprocessor

Major milestone achieved: JyNI builds successfully on Windows, DemoExtension is workable