Thursday, January 26, 2017

Creating Beauty with UglifyJS

For the past five years, we have been using a Notes-client base tool called iPhora Application Designer, that we created to build our iPhora applications written for our Dojo-based iPhora-MVC platform. This tool is written entirely in LotusScript and @Formula language.

With this tool we compile the  XML/JS view code into pure JavaScript onto a targeted application server all without opening Domino Designer.  Rarely do we venture into using Domino Designer. We only use Domino Designer to enhance our tool or fix a bug. Since this tool is Notes-client based, a key advantage of this tool is the ability to replicate every day our code to a backup server and when we are at home we can replicate to our computer at home.

Currently, our JavaScript MVC platform has over a 1,000,000 lines of Dojo/Javascript code and it grows every day as we add new widgets and modules. Majority of the code is automatically generated by our iPhora Application Designer. Can not imagine generating the code manually.

As part of the compilation process, we created a code HTML/JS minimifier in LotusScript to compress the code.  However, it never really worked the way that we wanted it to and it could not obfuscate the code. So we decide to look elsewhere for help.

There are many JS minimifier out there, but we wanted the ability to integrate it with our development process and also our iPhora Application Designer.  So we looked into integrating tools that we can run using the LotusScript Shell command to process the JavaScript.

After researching different alternatives, we decided to tried a node-based solution called UglifyJS which has a command line interface which we can invoke using the LotusScript Shell command.

First, you need to download and install the latest version of Node.  Make sure that you include the NPM manager during the installation process.

https://nodejs.org/en/download/

After installing node and npm, open a command line window and install uglifyJS from the NPM depository using the following command line.


npm install uglify-js -g

Now that you have installed UglifyJS, you need to confirm that it is working correctly. Create a simple JavaScript file called test.js and run UglifyJS using the following command.

uglifyjs test.js -c -m -o test.min.js

-c : compress the code
-m : mangle the code
-o : declare that there is output file with the following name

If the resulting test.min.js is a compressed mangled version of test.js then it is working.  

We could manually create files and then uglify them one at a time and then uploaded them to the server, but that is time consuming and laborious. Automation is key to reducing the time that you need to spend coding. So we needed to figure out a way to automate the process.

Our XML/JS code are stored in Notes Rich-text fields. When the code is compiled, the generated compiled code is pushed into the targeted Domino server using good old DXL. 

In order to minimify the code using UglifyJS we needed to change our process. Rather than compressing the code using the LotusScript and then pushing it up to the server, we needed a way to get it into the format that the Node-based uglifyJS can use, compress and mangle the code, read the code and push it into the server all in one single operation.  Show below is the process and code that we created to achieve our goal.

Read JavaScript Code from Rich Text

To get the code from the rich-text field using "GetUnformattedText()" Lotusscript rich text item method.

Set ritem = doc.GetFirstItem(fieldName)
If Not(ritem Is Nothing) Then
code = ritem.GetUnformattedText()
else 
code = ""
end if
fileName = tempGetNotesDirectory() & |\| & |input.js|
Dim fn%
fn = FreeFile
Open fileName For Output As fn
Print #fn,code
Close fn

Create Batch File to Process the JavaScript

cmdLine = |uglifyjs input.js -c -m -o output.js|

Create a batch file that has the command to run UglifyJS, UglifyJS.bat


cmdLine = |uglifyjs input.js -c -m -o output.js|

fileName = tempGetNotesDirectory() & |\| & |UglifyJS.bat|

Dim fn%

fn = FreeFile
Open fileName For Output As fn
Print #fn,cmdLine
Close fn

Run the batch using the LotusScript Shell command to create the file "output.js"

batchInstance = Shell("UglifyJS.bat",0);

The problem that you have running a batch file using Shell is that there is no way to know if the process has been completed using LotusScript. However, an easy way to check if the process is done is determine if the file output.js has been created.  If not, wait and check again. The following code allows you to query the file system for completion. The uglifying process is very fast, but a large JavaScript file may take longer. The code waits for 1 second and checks the file system. It tried this 10 times and if it can't find it the entire process is killed off.

Function runBatch() as Boolean
Dim tempdir As String
Dim i As Integer
i = 0
On Error GoTo ProcessError
batchInstance = Shell(tempdir &"\node_batch.bat", 6)
Do While i < 10
Sleep(1)
If Dir$("output.js",0) <> "" Then
Exit Do
End If
i = i + 1
Loop
If i > 10 Then
runBatch = false ' process timeout
Else
runBatch = true  ' process ran successful
End if

End Function

After the process completes we need to read the output.js file so that we can push it up to the Domino server.

Function readJSFile() As String
Dim tempdir As String
Dim fileName As String
Dim uglifiedCode As String
On Error GoTo ProcessError
tempdir = tempGetNotesDirectory()
fileName = tempdir & |\output.js|
Dim fn%
fn = FreeFile
Open fileName For Input As fn
uglifiedCode = Input(LOF(fn), fn)
Close fn
readJSFile = uglifiedCode
End Function

Using our DXL-based methods we stream the compressed and mangled file up to the Domino server.

There is one more step that is required. We need to delete all the files that we created in the temporary directory so that we can process another file. If not, the process can not be repeated since there will always be a output.js file.

Function removeTempFiles() as Boolean
tempdir = tempGetNotesDirectory()
kill tempdir & |\uglifyjs.bat|
kill tempdir & |\input.js|
kill tempdir & |\output.js|
End Function

tempGetNotesDirectory(), function to find the Notes temporary directory

The inclusion of UglifyJS to our iPhora Application Designer has been so successful that we have added other Node-based developer tools to it using the same method to connect our Notes-based tool to new and modern development tools. So far we have added CSSLint and Minifier to cleanup and compress our CSS file. We are looking for other Node modules that we can include to help us automatic the development and testing process. There are thousands of npm modules that run on Node. There are many to choose from.

If you are using the Domino Designer, you should look at this Node-based tools to add new capabilities in your developer's toolbox. How that can be done is beyond my skills.




Wednesday, September 14, 2016

The Future of Domino from the POV of an ISV

Many in the ICS community have made the statement that Notes and Domino are dead.

Many in the ICS community have been questioning the future of Domino and they have ever right to do so considering the lack of and inconsistent communication from IBM and lack of improvements to the platform for the past 4 years.  Yes we have had improvements regarding SSL security fixes but that was driven by the discovery of serious vulnerabilities in SSL.

Like many customers and IBM partners, we built many many Notes applications internally and for clients. Some of these applications are still running on many of our clients and will continue to do so for the foreseeable future. They do not have the coolest interfaces since those capabilities weren't around even from other vendors at the time they were developed. They are clunky looking, but they are secure which is very important since the beginning of Notes and more so these days. 

IBM announced yesterday that they are extending support for Domino at least until 2021 and will be providing incremental enhancements to Domino through fix and feature packs which I am glad to hear. Personally, I dislike the naming convention for future releases which will be provided as feature packs. I thought it was confusing and did not project the feeling that Notes and Domino were vibrant products. IBM should start with a different naming convention given that they are switching to incremental enhancements through feature packs.

Over 5 years ago, we started making the transition from a consulting firm to an ISV. As an ISV that provide business solutions either through mobile or web client, the naming convention is irrelevant for us. Customers never sees the version number nor they even know that it is Domino at the core.

There are three types of ISVs for Domino. Looking at our wonderful MWLUG sponsors, you have application ISVs like Riva CRM Integration who develop applications that works on Domino, you have IBM ICS product ISVs like BCC, Crossware, HADSL, panagenda, RPR Wyatt, Teamstudio and Ytria that enhances and augments the ICS product portfolio, and then you have us, Phora Group. Why do we separate ourselves from other ISVs?  

We wanted a scalable multi-tenant application architecture with data security being the up-most of importance. For Phora Group, Domino is a JSON-based NoSQL database infrastructure comparable to MongoDB, CouchDB and Couchbase but data security built into it core architecture. It is an application/database server.  When we started the process of developing iPhora, we looked at developing it using these other technologies.  However, the advantage of Domino is that authentication, data distribution, web services, integration, data security, application logic and many others things are built into its architecture all packaged into a small but scalable footprint. There are stuff in Domino that just isn't that easy to duplicate using other platforms.  

When we designed iPhora, we wanted its architecture to be database infrastructure agnostic and if needed, we could transition it to other database technologies.  So we developed the iPhora architecture as abstraction layers with application logic driven through RESTful services that are surfaced with a JavaScript-based MVC. The architecture is based on what many call "modern" or "cooler" approaches. We redesigned many of the functionalities in Domino including security and data management all of which can be transitioned to different platforms. The core components found in Domino are what is important to us and is what makes Domino unique.

So from our unique POV as an ISV as long as Domino is alive and kicking it is good news for us. Based on what we are hearing from IBM, the core Domino technology will continue to be an integral part of our iPhora solutions. It provides us a unique platform unlike any other, even after 26 years.  My wish list is not more features, but separate Domino into core blocks and allow us ISVs to build Domino like Lego blocks to match our needs. Less is more.

Wednesday, August 3, 2016

A Welcome Message to Austin from our IBM Community Manager

It is two weeks away from the start of MWLUG 2016.  So get registered for this last ICS conference event in the United States for this year.


http://mwlug.com/mwlug/mwlug.nsf/register.xsp


Thursday, July 28, 2016

2016 Top Ten Reasons Why You Should be Coming to MWLUG

Here is the 2016 Top Ten Reasons why you should be coming to MWLUG:
#10: 52 technical and business sessions with great speakers
#9: Why the ICS community is one of the best technology communities on this planet!!!
#8 Bats, lots and lots of bats
#7 What color suit coat will Bill Malchisky wear to the conference
#6 Simplicity BEER!!!
#5 MUSIC
#4 Why Kim Greene is the true Drinking Champion of them all
#3 Watch Richard Moy mess up his words at the OGS because he never scripts or practice his speeches
#2 Whether Justin Hill loses his bet again

. . . And the #1 reason to come to MWLUG 2016, learn whether Darren Duke will take off his shirt at the Thursday Evening Event again?


So get registered and we will see you in three weeks

Monday, July 25, 2016

MWLUG 2016 Session Schedule is Now Available

The session schedule for MWLUG 2016 is no available online at:

http://mwlug.com/mwlug/mwlug.nsf/schedule.xsp

There is still time to register for MWLUG 2016 at:

http://mwlug.com/mwlug/mwlug.nsf/register.xsp

Friday, July 22, 2016

Coming to MWLUG 2016, Learn What Collaboration is all About, Thursday Social Event

We are please to announce the MWLUG 2016 - Thursday Social Event will be held at Zilker Brewing Company, rated as one of the best breweries in Austin, TX. This private event is open to all registered MWLUG 2016 attendees. Attendees that are interested in a tour of the brewery are welcome to join us during the event. We will be providing a bus from the Four Seasons Hotel to Zilker Brewing Company and back. MAKE SURE you have your MWLUG 2016 badge when we board the bus and at the event at Zilker Brewing Company. We will be providing food from a local establishment yet to be announced.
So come join us and register for MWLUG 2016.
http://mwlug.com/mwlug/mwlug.nsf/register.xsp



Tuesday, July 19, 2016

Announcing the MWLUG 2016 OGS IBM Speaker

We are please to announce the MWLUG 2016 Opening General Session speaker is Katrina Troughton, General Manager, Social and Smarter Workforce Client Success. Katrina brings a unique perspective our ICS community.





Katrina Troughton is General Manager, Social and Smarter Workforce Client Success. Katrina leads IBMs worldwide sales force for collaboration, social, exceptional digital experience, and smarter workforce solutions.
Katrina has more than 23 years' experience in the IT industry, with more than 20 years spent in senior management roles at IBM. Prior to leading this sales team, Katrina led the Business Analytics team for IBM's Growth Markets Unit, IBM's WebSphere business for IBM's Growth Markets Unit, both, after spending four and a half years as Managing Director for IBM New Zealand. Other key roles include Software General Manager for IBM Australia/New Zealand, and sales executive for Information Management for IBM Asia Pacific.
Katrina has served on a number of external boards including New Zealand Advisory Board for Global Women, and Uniservices (the commercialisation arm of Auckland University); additionally, she chaired the NZ Government Digital Strategy Advisory Board (2006 2008).
Troughton holds a BSC / BCom, with majors in Pharmacology, Physiology and Economics, from Auckland University.

Thursday, July 7, 2016

MWLUG 2016 IBM Design Tour Registration is Now Open, Wednesday August 17, 2016

As with our 2016 theme "Defining the Art of Collaboration," the MWLUG 2016 conference brings everyone together to share knowledge and expand awareness of innovations in IBM collaboration community, thus building and growing our community.

For MWLUG 2016, we have arranged a special private tour for all conference attendees of the IBM Design Center in Austin, TX for Wednesday, August 17, 2016.  This tour is limited to the 120 attendees only.  We will be providing a bus to bring attendees to the IBM Design Center.  The first bus will leave the Four Seasons Hotel in Austin at 8:00 AM and will continue to drop off and pick up attendees to the IBM Design. The last bus will leave the Four Seasons at about 10:45 to 11:00 AM. The last returning bus that will arrive back to the Four Seasons at approximately 1:00 PM.

You must be a registered attendee by the time you board the bus and have your MWLUG 2016 conference badge to participate in this event.  Attendees who sign up for this event, will be given their MWLUG 2016 conference badge before boarding the bus. The bus has a capacity of 40.  Each tour has the capacity of handling a maximum of 20 attendees at one time and last 20 minutes.  There is no cost for this event.  The cost of the bus is paid by the generous MWLUG 2016 sponsors.  So thank them when you see them.

To register:
https://www.eventbrite.com/e/mwlug-2016-ibm-design-center-tour-tickets-26468340519


mwlug 2016 sponsors