Mindoo Blog - Cutting edge technologies - About Java, Lotus Notes and iPhone

  • New APIs for Domino JNA project, now available for XPages development

    Karsten Lehmann  16 January 2017 08:21:26
    It took some time (mainly because I was very busy and my main use case for the project is not XPages), but now the Domino JNA project is available for XPages development.

    That means that there is an XPages Extensibility API plugin available in the project's release section on Github that can be installed both in Domino Designer and on the Domino R9 server to use the API in your own applications.

    After installation, the API classes are available in SSJS and Java code including API source code.
    Since they are integrated with the JSF lifecycle, memory management is automatically handled by the plugin. That means that code using the API does not have to be wrapped in NotesGC.runWithAutoGC() blocks to automatically recycle allocated C handles and memory after a HTTP request.

    If you want to build your own release of the project (or of your own fork), please take a look at the section "Using Domino JNA in XPages applications". The Maven build has been tested on macOS and Windows.

    New features

    Tons of new features have been added to the project during the last months. It's difficult to keep up with test cases and extend code documentation.

    Here are a few highlights:
    • new APIs to use native NSF transactions (atomically create/modify/delete multiple documents in a database, see the Transactions class for known issues/limitations)
    • new APIs to read the ECL for a specific user
    • API to let NSF quickly filter a number of documents with a formula (not just the whole database as in IBM's Java API): NotesIDTable.filter(NotesDatabase db, String formula)
    • support added to direct attachment streaming to quickly skip a number of bytes (can be used to implement partial downloads via Accept-Ranges HTTP header, restricted to uncompressed attachments)
    • APIs to read/write database options with more than 100 available option constants (IBM's Java API contains only 13 of them)
    • Reading selected view rows specified as an ID table now also works on remote servers (use NotesCollection.updateFilters() to push your SelectedList ID table changes to remote servers before reading data)
    • Performance enhancements when inserting many IDs to a NotesIDTable

    To achieve the new functionality, lots of undocumented C API methods have been used that IBM should really add to the public API:
    • NIFReadEntriesExt
    • NIFIsNoteInView
    • NIFIsUpdateInProgress
    • NIFGetIDTableExtended
    • NIFCollectionUpToDate
    • NIFUpdateFilters
    • NIFIsTimeVariantView
    • IDScanBack
    • IDInsertRange
    • IDTableDifferences
    • IDTableReplaceExtended
    • ECLGetListCapabilities
    • NSFSearchExtended3
    • NSFDbGetOptionsExt
    • NSFDbSetOptionsExt
    • NSFTransactionBegin
    • NSFTransactionCommit
    • NSFTransactionRollback
    • extended FIND_XXX flags e.g. FIND_CATEGORY_MATCH to look up category rows in views
    • more than 100 DBOPTBIT_XXX constants to read/write database options

    Please let me know if there are any problems using the API.

    Comments

    1Michael Bourak  16.01.2017 9:50:22  New APIs for Domino JNA project, now available for XPages development

    Karsten for President !

    More seriously, this projet is simply amazing !

    It can truely change your life as a modern domino developer and make some nearly impossible things a snap.

    Really, Really, Thanks a lot !

    2Stephan Koops  16.03.2017 10:31:15  Is possible to run not as OSGI-Pluign?

    Hi, this project looks very cool.

    When you want to use Domino JNA in Java Code from XPages, it is required to be added as OSGI plugin, or could I put the JARs and so on into a nsf?

    Do you now if I have to change the java.pol for JNA native.

    3Karsten Lehmann  16.03.2017 10:37:19  New APIs for Domino JNA project, now available for XPages development

    Hi!

    I think Sven Hasselbach added the Domino JNA JAR and its dependencies to the NSF before I had finished the plugin build process. He described the steps here:

    { Link }

    I would recommend using the plugin because it makes your code simpler. For example the code does not been to be wrapped in NotesGC.runWithAutoGC blocks, because I already do this in the plugin for each HTTP request.

    And I am sure that you need to add more lines to java.pol than Sven mentioned in his post. That should be automatically handled by the plugin code via AccessController.doPrivileged calls in various places.

    4Stephan Koops  16.03.2017 10:58:43  Re: Domin JNA in nsf

    Thanks for the quick answer.

    One thing, is what the developers want, and the other thing is, what the admins of our customers say, e.g. "no OSGI" or "no changes in java.pol". For the last stuff we could implement the slow way with Notes.jar as alternative.

    Thanks for the hints about NotesGC.runWithAutoGC and the doPrivileged.

    5Karsten Lehmann  16.03.2017 11:38:31  New APIs for Domino JNA project, now available for XPages development

    I am not sure if its possible to use Domino JNA in multiple databases that way, because AFAIK the JNA dll can only be loaded once per JVM. In that case, you would need to copy the jna.dll to jvm/lib/ext so that it gets loaded by the JVM base classloader.

    6Stephan Koops  16.03.2017 16:21:29  Thanks for the next hint

    Thanks for the hint about jna.dll. We will have a look.

    7Stephan Koops  04.04.2017 10:55:03  Could not read entries form some views, other views have no problem.

    Hi,

    I want to read view entries with Domino JNA, in the first step with getAllEntries(). With the Notes.jar I get always the expected results, but with Domino JNA I giot sometimes no results. It works fine with your example database (fakenames.nsf), also with my local names.nsf, but I have problems with self created database with self created views, created with Java API or with Domino Designer. I tried databases created with Notes.jar-API and with the Domino Designer. Do you have any ideas?

    8Stephan Koops  04.04.2017 11:05:26  Re: Could not read entries form some views, other views have no problem.

    I forgot: The code is started in a JUnit test.

    9Stephan Koops  04.04.2017 11:22:05  Re: Could not read entries form some views, other views have no problem.

    Now I get it working after two days of trying around: I removed all reader / author items. Now the question is: Why could I read exactly the same documents with Notes.jar, but not with Domino JNA?

    10Karsten Lehmann  04.04.2017 11:33:58  New APIs for Domino JNA project, now available for XPages development

    Hi!

    I am not sure what is causing this issue.

    The NotesDatabase class has several constructors to decide which identity you want to use to open the database.

    E.g.

    NotesDatabase(Session session, String server, String filePath)

    =uses the effective username of the session object and lets Domino compute a NAMES_LIST structure to be used to open the DB

    NotesDatabase(Session session, String server, String filePath, String asUserCanonical)

    =uses the specified Domino user and lets Domino compute a NAMES_LIST structure to be used to open the DB

    NotesDatabase(Session session, String server, String filePath, List<String> namesForNamesList)

    =we write our own NAMES_LIST structure here, so you can combine any username/wildcard/groups/roles you like to let Domino check for reader/author access

    and finally

    NotesDatabase(Session session, String server, String filePath, (String) null)

    =opens the database as the server

    There was a memory alignment bug in 0.9.2 and earlier versions for Linux/64 that might prevented the method from working properly, but I would expect an ERR 582 on database open instead of not returning all data.

    I fixed that in the latest 0.9.3.

    11Stephan Koops  04.04.2017 13:48:33  Re: Could not read entries form some views, other views have no problem.

    FYI: I had a detailed look into the names fields: There were empty reader fields (empty means ""): Maybe Notes.jar also add's "" to the namesList.

    12Stephan Koops  04.04.2017 13:48:53  New APIs for Domino JNA project, now available for XPages development

    I have some improvement proposals:

    * use for NotesCollection the name View. Evereybody know it with this name. And every Java developer will search for database.getView(viewName) or openView(viewName).

    * some pom.xml improvements to reduce dependency to operating system

    * improve exception message: Give hint what failed in NotesErrorUtil.checkError()

    * if you use interfaces, you have the possibility to let the most stuff fall back to Notes.jar, e.g. when an application runs on a supported envirnoment and on an unsupported environment.

    I also have the problem that some JUnit tests fail in the notes client.

    Should I create a pull request? I think it is useful, when we have a skype call to have a look into the code and have maybe a small discussion about it before creating the pull request.

    13Stephan Koops  04.04.2017 17:00:06  does Domino JNA detect unsupported environment?

    Does Domino JNA already detecty automaticly if it runs in an unsupported environment like AIX? Otherwise I will add it.

    14Karsten Lehmann  04.04.2017 17:02:26  New APIs for Domino JNA project, now available for XPages development

    No, I don't think so.

    com.mindoo.domino.jna.internal.NotesJNAContext.getNotesAPI() would be a good place.

    Let's continue this in the issues list of the Github project. That makes it easier to track progress on todos and we have code formatting.

    15Patrick Kwinten  03.10.2023 12:31:07  New APIs for Domino JNA project, now available for XPages development

    Unfortunately, when trying to open a notesdatabase in xpages it stumbles at the following:

    NotesDatabase db = new NotesDatabase(ExtLibUtil.getCurrentSession(), "", fakenamesPath);

    in log.nsf I read:

    HTTP JVM: java.lang.IllegalStateException: Thread is not enabled for auto GC. Either run your code via NotesGC.runWithAutoGC(Callable) or via try-with-resources on the object returned by NotesGC.initThread().

    I am running the 0.9.48 osgi plugin version.

    It seems that memory management is NOT automatically handled by the plugin.