OFAC/SDN Watchlist Software Application Axiom #6

While sourcing watch list consider a watch list provider like www.world-check.com. Rather than manually collecting watch lists from different agencies like OFAC or Bank of England, the software solution which automatically fetches watch lists (via ftp or some other method) is much more reliable and scalable. Keep the manual uploads to your watch list scan application as minimal as possible.

OFAC/SDN Watchlist Software Application Axiom #5

Watchlist Scanning applications which have the capability of reporting “alerts” should consolidate alerts arising from the same “Entity” from different watch list as 1 alert rather that reporting the same entity multiple times.

This will eliminate duplicate work as multiple analysts can work on the same entity if she is reported more than once due to the fact that she is on different watchlists.

Database Metadata Information Leverage

Cannot stress the importance of an upto date database metadata information for an application or a suite of applications which exchange data amongst each other.

Metadata information can be leveraged in following ways :

- Impact Analysis
Application sourcing data from upstream application and/or sourcing data to downstream application can hugely benefit by having a metadata repository. It will enable the application owners to easily identify the producers and consumers and hence enable them to perform impact analysis.

- Data Reuse
As a byproduct of meta data harvesting exercise, the application owners will easily be able to identify redundant data (data which is sourced more than 1 once for a suite of applications)

- Documentation
Rather than dedicating time to create database documentation, any metadata tool will enable you to export html and publish it on your intranet possibly on sharepoint or wiki site which are easily searchable.

- Consolidation of Load Process
In an organization having a suite of applications, more often than not there are disparate data load batch processes executing to fetch data (from data warehouse let’s say). Metadata harvesting exercise will enable the application owner to identify and potentially consolidate the batch process on a single platform (etl tools like informatica)

Count Unique values in a list - Microsoft Excel

To count unique values amongst a list of values, click here.

I am copy pasting the first suggestion here only to avoid future link failures.

Let’s say you want to find out how many unique values exist in a range that contains duplicate values. For example, if a column contains:

* The values 5, 6, 7, and 6, then the result is three unique values—5 , 6 and 7.
* The values “Buchanan”, “Dodsworth”, “Dodsworth”, “Dodsworth”, then the result is two unique values—”Buchanan” and “Dodsworth”

1. Ensure that the first row in the column has a column header.

2. On the Data menu, point to Filter, and then click Advanced Filter.

3. In the Advanced Filter dialog box, click Copy to another location.

4. If the range that you are counting is not already selected, delete any information in the List range box and then click the column (or select the range) that contains your data.

5. In the Copy to box, delete any information in the box or click in the box, and then click a blank column where you want to copy the unique values.

6. Select the Unique records only check box, and click OK.
The unique values from the selected range are copied to the new column.

7. In the blank cell below the last cell in the range, enter the ROWS function. Use the range of unique values that you just copied as the argument. For example, if the range of unique values is B1:B45, then enter:
=ROWS(B1:B45)

Improve HTML page loading time

I cannot stress enough the importance of compression while loading html pages. One of the web applications (struts, weblogic implementation) I worked on was experiencing slowness while loading a search results page. The page used to take about 4 seconds to display (assume pure static content for the sake of this discussion) The page size was 638K! and there was no way I could reduce the amount of “visible” information.

Couple of things I observed and implemented which really helped cut 2 seconds of the page loading time. I am sure a lot is infrastructure dependent i.e : how stressed the network is etc. but there are things which are under developer’s control which we can leverage to improve page loading time.

Observation #1 :
The jsp had a for loop, which contained td style=”font-family: “Trebuchet MS”;font-size: 12px;color: #797b7a;font-weight: normal;” …. /td

I. e. : if the loop iterates for 500 times, that is the number of times “font-family: “Trebuchet MS”;font-size: 12px;color: #797b7a;font-weight: normal;” gets repeated.

Simple Solution :
Put this in css, refer that css in your jsp and change the td to . Huge reduction in generated html size. Instantly!

Point to note is : less number of characters used in the html (especially in a for loop) the better it is.

Observation #2 :
The page after it was built was not compressed. Even after the above change, the page was about 350K.

Simple Solution :
All browsers now support gzip compression. Add a filter at web.xml level and compress your html content before it hits the wire. A *huge* size savings here. The page size was reduced to 17K! while over the wire.

The effort to compress at the server level and decompress at the browser level takes much less time as compared to transmit 350K over the wire.

Many thanks to Jason Falkner for his excellent article on this. Completely agree with him to make the compression as part of a must-do for any web application

Case Closed letter from IRS for Non Profit Application

As from my other posts on non profit, you would have noticed that I did the paperwork neccessary for non-profit (501c3) setup with IRS myself. The application was submitted in August ‘07 and was approved in April ‘08.

However, in January ‘08. I received a letter from IRS stating that they had tried to contact me at my mailing address and since I had not responded before the end date mentioned in that letter, they had closed my case. Needless to say that it was a frusterating moment for me because I had never received a letter from the IRS and I was in constant touch with them since December ‘07 and was assured that my application has been assigned and the agent is working on it.

After a couple of phone calls and snail mails my case was reopened and was finally approved in April’ 08.

So, if you get a similar letter from IRS, don’t be disheartened, you will have to just present your case calmly to the assigned agent and s/he will work with you to get it reopened and approved. The IRS agent I was working with was very helpful and guided me thru the whole process.