JwContacts (JwC) is an independent effort by one Jehovah’s Witness to address certain otherwise unmet needs Witness Elders often face. For the rationale in developing it, please see the blog posting “Why JwContacts?”
While still employed I began to study the advantages of what was then the brand-new version of the common Internet presentation framework, HTML 5. It seemed possible to provide a Territory composing application and work began, even to the point of asking some local brothers to help me form a non-profit company to develop it. However, it became apparent the time limitations of secular work and theocratic responsibilities precluded continuing.
By the time of retirement in 2016 others had achieved the kind of territory help I had envisioned. What other need might be addressed? A quick web search found many applications to assist in general field ministry and in working with territories. The excellent Kingdom Hall Schedules (KHS) from Joe Majestic had been used with great appreciation for many years. What was left? Nowhere did I see help with something that had bothered me for years: addressing quick, private access to publisher contact data under the broadest range of circumstances. Ever more we were hearing about Elders’ need to offer assistance to those affected by emergencies and disasters; could I help in some way? JwC is my answer, pending some similar help via the Governing Body.
In 2016 Microsoft had just acquired Xamarin, a means to write applications that were “cross-platform”, working from a common code base in C# to generate native software for various platforms, such as Android, Apple, Windows, etc. The variation Xamarin Forms (XF) added the ability to have that same base also define the general visual appearance, which Xamarin would then automatically personalize for each supported platform. I had thought about Xamarin for some time, but could not afford it. Lo, Microsoft in 2016 starts giving it away!!
As I had never previously done any Android or Apple programming, much less any Xamarin development, the rest of 2016 (and some time since) was spent in study of Xamarin and, later, in some understanding of the languages that earlier had been used in Android and Apple programming. Spending much time considering what sort of database schema (the way data would be organized), I gradually defined the earliest and by early 2017 began to actually code.
Very shortly I had done enough to show proof-of-concept. Our Circuit Overseer then I had known since we were both young and he was among the first shown. He suggested I write the Branch about my intentions and see what might happen. This was done but no answer was ever received.
A few weeks later I moved a short distance theocratically to our current Congregation, highly enthused about my new project, finally getting underway. Earliest attempts at demonstration resulted in hangs of the infant, buggy code; reception was extremely cool. We all would have been better served by my waiting until code was nearing maturity.
Meanwhile JwC has been getting steadily fleshed out. Thousands upon thousands of hours (doubtless an exaggeration, but you get the idea) have been spent in bringing to life the various Elder aids JwC now includes. But, in late 2017 another surprise: Microsoft changed its definition of acceptable Xamarin programming. ‘Til then the common code might be in either (or a combination) of two forms, a Shared project (shproj) whose contents would be replicated in other projects using Compiler directives to define those portions peculiar to one (or more) platforms but not the rest AND / OR Portable Class Libraries (PCLs) whose code was usable in a defined subset of all then available platforms. PLCs constituted a recipe for creating self-obsolescence as new platforms emerged and was abandoned in favor of the new .Net Standard Library.
Microsoft had by then for a few years been working on related structures termed .Net Core and .Net Standard, both meant to facilitate development for both Microsoft and non-Microsoft platforms. .Net Standard is discussed in the Technical Overview blog, but is what its name suggests: A standard that is expected to be fully implemented in ALL compliant platforms, with each platform designer creating the actual implementation. There were a long series of early .Net Standard 1.x versions, the last being 1.6.1. But industry agreement led to .Net Standard 2, first implemented in Microsoft’s own products beginning September 2017. There have been a few later refinements; the current version is 2.0.3.
Coming in a year or two is another similar “earthquake”, the arrival of .Net Standard 3.0. At this point I have no idea what a .Net Standard 3.0 version of JwC would be able to support. Time will tell.
However, as it was apparent all new Xamarin software would be written for .Net Standard 2 compliance, fall of 2017 brought a complete rewrite of all JwC code. JwC is now nearly 100% .Net Standard 2 compliant, save two third party modules still at just 1.6.1 compliance.
Another “earthquake” was growing appreciation for the need to secure all JwC data from eyes not entitled to it. The spring 2018 arrival of the European GDPR (General Data Protection Regulation) only confirmed need for another sweeping JwC change. All mobile platforms have long inherently supported a self-documenting data storage mechanism know as SQLite. At least at first JW Library appears to have used SQLite, as did JwC. The availability of free SQLite tools and dozens of third-party SQLite aids had seemed to make SQLite a “no-brainer” choice. A SQLite database’s schema could very easily be updated without having to re-install the entire app. Unfortunately, not only does SQLite have no provision for encryption, its self-documenting nature allows anyone who can get hold of an app’s copy to readily discern its schema.
A search found the free REALM database had the requisite security features. Hundreds of hours were spent in redefining JwC’s data schema for REALM and in providing optional REALM encryption. But REALM also brought some headaches, chief of which was the extreme difficulty in instituting any schema change. For many months even the slightest schema change meant completely removing the old JwC, its user-supplied settings, and all its data, then installing the new, re-installing the data to conform to the new schema, and have the user re-enter all his settings. Ughhhhhh! The hours thus expended in making any very slight schema change are almost beyond counting!
Finally, having had enough of this and seeing that months of imploring REALM’s providers for a solution that actually worked in Xamarin had not been fruitful, in late 2018 I decided to “bite the bullet” and make the change. The REALM code, though publicly available, was in a form beyond my comprehension (or willingness to study sufficiently) and I gave up thoughts of amending it. What to do?
Although I hated bloating JwC installs any further, I decided to create a hidden backup folder for all REALM data, creating encrypted XML of each type, with a “table of contents” file possessing what JwC would need to unencrypt. Encryption / un-encryption keys are specific to a device and will not work on other devices. Every time JwC “goes to sleep” at User request these files get created just before app end occurs.
This encryption was based on work done previously to allow data sent out of JwC to be encrypted.
JwC now contains a static version code of its REALM schema, which code is stored in Xamarin secure data persisting between app launches. If ever JwC launches with a changed version code, signifying a new JwC update containing a REALM schema change has been installed, on launch JwC immediately destroys all existing REALM data and then re-installs it properly from the encrypted XML backups. Newer JwC releases with new REALM schema version codes will understand how to do this without data loss.
This scheme was submitted to the REALM community for use with Xamarin and was accepted for sharing.
Many would-be users expressed the desire to avoid data sharing and backups via email by instead working with remote data storage services like Dropbox. This sort of thing remains poorly documented in Xamarin sources, with the exception of a Microsoft tool called Xamarin.Auth (XA). XA assists in getting a remote service to recognize a user wants to allow an app to access his storage. Getting such access is perhaps the most difficult step, but is only the start. Most services follow an authorization / access granting procedure known as OAuth 2.0 (OA2), but the procedure allows each service great flexibility in its own implementation.
Some services (Dropbox and Google Drive) work well with XA’s method of meeting their OA2 requirements, others hardly at all. And while I could find useful examples of XA use in Android and Apple devices, Windows examples have proven elusive until recently. For now, JwC solely uses XA to gain access to just Dropbox and Google Drive on Android and iOS devices.
But even access is not enough. File listing and transfers require detailed knowledge of each service’s API (Application Programming Interface); no two service’s have even similar APIs. Fortunately, both Dropbox and Google have APIs understandable to programs using C#, as is the case with Xamarin Forms. Google even provides Xamarin tools to use many of its services, including Drive. And one Xamarin programmer took the time to make a Xamarin-compatible Dropbox API. So JwC has been able to provide consistent, reliable use of both Dropbox and Google Drive.
Good, but not enough. Windows users may well prefer OneDrive from Microsoft. Apple users might prefer iCloud Drive with its just-announced new features of file sharing coming shortly. And there is a plethora of other remote services. What to do?
At first it seemed a free intermediary service called Cloudrail would be the answer. At the moment JwC’s use of Cloudrail has allowed good authorization / access-granting / file transfers with OneDrive and a service called pCloud. Cloudrail also offered ability to work with Box and Egnyte, but tests did not pan out.
In March, 2019 Cloudrail announced ceasing further development and technical support of this aspect of its business, although the service continues to operate, for now. Tomorrow it may vanish, who knows? A search for alternatives has been almost futile, although Kloudless (not entirely free) may prove useful someday. Kloudless supports some of what Cloudrail did and adds other services.
But dependence on intermediate services of this sort makes JwC fragile, as Cloudrail has shown. So, work began on preparing C# APIs for Box and Egnyte, as these supply sufficient public details. Box ALMOST works with XA and seems popular enough future effort ought to be expended to eventually have JwC provide native support. Egnyte is meant for teams, not individuals, is never free, and few JwC users would likely make use. Still I may eventually get past the authorization / access-granting hurdle for both Box and Egnyte. The real need for each is that I may have to set up my own intermediary on the JwContacts web site.
I happen to really like pCloud, but its only public APIs are unlike C# and would require a lot of computer language translation. Before Cloudrail goes away I would like to support pCloud natively, but I’m not sure I’ll ever have the time.
As to Microsoft’s OneDrive, free to every Windows user and accessible on all devices, its OA2 implementation is totally unique. I am very thankful Cloudrail works well with it and dread the day Cloudrail goes away. I have no idea how much effort I’d have to expend to provide native support; perhaps sometime soon some other programmer will release a sample. OneDrive is supported by Kloudless, but I would prefer not yet going that route.
At this point iCloud Drive support is just a gleam in this programmer’s eye. The promised future iCloud enhancements will certainly make use desirable; we’ll see where we go from here.
So we’ve covered a lot of territory but primarily concentrated on three subjects: (1) Adapting to changes in Xamarin programming, a still-evolving technology, (2) replacing the original SQLite database with the encryptable REALM database while dealing with the issues REALM added, and (3) providing use of remote storage services for JwC users who prefer them to email use. That’s enough for one blog; other less gut-wrenching matters may be addressed in future blogs.
Thank you for your support and interest. Please feel free to comment / get in touch.
David Pressman
