Skeptical Inquirer

Pursuing the ironic debunking of commercial myths and accidentally creating some others. Copyright reserved by the Authors.

Tuesday, November 17, 2009

Updating ListView in a ListView with an Ajax behaviour

I have a list of questions that I format in a template in a two dimensional array. Why? because this stuff is going to be printed out and therefore needs to neatly fit on a page before being iTexted into a pdf/rtf or excel spreadsheet.

The objective is to use Ajax links to update data in ListView without forcing the whole page to be reloaded from the server (why? firstly, because your page will be loaded at the top and you will lose your position on the page, and secondly, ajax is a faster user experience because less data is sent back and forward).

But in order to do this, we face two major challenges:
  1. We need to find a target for our AjaxRequestTarget to repaint so that our ListViews can be refreshed to reflect any changes to the list eg. if we changed the question order.
  2. We need find a way to update the underlying data for the ListView while also updating our domain objects to persist the data in the database.

But in the spirit of OO development, we have been embedding panels in panels and suddenly we need to get a handle on our rendering container for re-painting while also getting hold of the ListView data (by calling listview.getlist()) for updating.

So the strategy is:
Step 1
Firstly, add all our listviews to a WebMarkupContainer in order to target it for re-rendering (Listing 1 and 2). Note that we are passing the container into the subpanels because we need to keep track of it for re-rendering. It is also possible to retrieve this container by ID see Listing 4.

public class QuestionEditPanel extends QuestionRenderPanel {

    public QuestionEditPanel(String id, final TemplateWebModel templatewebmodel) {
        super(id);
        Listinitial=templatewebmodel.getEntity().getQuestions();
        final WebMarkupContainer container=new WebMarkupContainer("container"); //container that will hold our listviews for repainting
        container.setOutputMarkupId(true);
        ListView rowslistview = new ListView("rows", QuestionProcessor.getListToMatrixWithLF(initial)) {

            @Override
            protected void populateItem(ListItem item) {
                List row = (List) item.getModelObject();
                ListView rowlistview = new ListView("row", row) {

                    @Override
                    protected void populateItem(ListItem item) {
                        final QuestionBase question = (QuestionBase) item.getModelObject();
                        item.setModel(new CompoundPropertyModel(question));
                        QuestionBaseWebModel questionmodel=new QuestionBaseWebModel(question);
                        item.add(new EditableQuestionPanel("question", questionmodel,templatewebmodel,container)); //note that the container is what we will repaint
                    }
                };
                item.add(rowlistview);
            }
        };
        add(container);
        container.add(rowslistview);
    }
}

Listing 1 QuestionEditPanel.java


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns:wicket>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
        <title>QuestionRenderPanel</title>
    </head>
    <body>
        <wicket:extend>
            <div id="document">
                <span wicket:id="container">
                    <div wicket:id="rows">
                        <span wicket:id="row">
                            <span wicket:id="question" />
                        </span>
                    </div>
                </span>
            </div>
        </wicket:extend>
    </body>
</html>

Listing 2 QuestionEditPanel.html


Step 2
Update both the domain model and the underlying data model for the ListView (Listing 3). Note the replication as you need to do the same thing twice (or you can update the domain model and retrieve the updated values to set the ListView model appropriately). So, to be clear, the ListView does not know when its data changes and therefore even if you update the Domain model, you still have to manually change the data and set the value on the ListView (Listing 4).



AjaxFallbackLink up = new AjaxFallbackLink("up") {

            @Override
            public void onClick(AjaxRequestTarget target) {
                AbstractDataSet dataset = datasetmodel.getEntity(); //domain object where we store questions
                dataset.moveQuestionUp(question); //move the question up on the domain object
                List list = QuestionProcessor.getMatrixToList(getRowListData()); //turn our matrix into a list
                QuestionProcessor.moveQuestionUp(list, question); //move the question up in the list
                setRowListData(getListToMatrix(list)); //set the list as the underlying data for the listview
                adsf.store(dataset); //store our domain model to not lose changes
                target.addComponent(outercontainer); //add our container for re-rendering
            }
        };
Listing 3 QuestionNavigationPanel.java (part 1)


private List getRowListData() {
        MarkupContainer container = getParent();
        while (!container.getId().equals("rows")) {
            container = container.getParent();
        }
        ListView view = null;
        if (container instanceof ListView) {
            view = (ListView) container;
        }
        return view.getList();
    }

    private void setRowListData(List data) {
        MarkupContainer container = getParent();
        while (!container.getId().equals("rows")) { //We traverse all the parents until we find one call "rows"
            container = container.getParent();
        }
        ListView view = null;
        if (container instanceof ListView) {
            view = (ListView) container;
        }
        view.setList(data);
    }
Listing 4 QuestionNavigationPanel.java (part 2)

In summary
Once this is done,  your AjaxFallbackLink will move a question up, down or delete it from a row, update the domain model so that that the changes persist in the database and then modify the model for the ListView. The ListView must be added to a container that is then re-drawn by the Ajax behaviour.

Monday, November 16, 2009

Open Service Definition

Now that the Open Source revolution is old hat, we look towards the bright future of cloud computing. Technically, OSS (Open Source Software) provided us with the right to modify code, run it ourselves, distribute it and still protect the rights of the producer to have the benefit of our improvements (and protection against unfair competition from us).

From a business point of view, OSS has had 3 major consequences:
  1. It provided an escape clause to vendor lock-in. If the client and vendor had a fallout over price/functionality etc. the client could move/fork/hire an alternative while still using the same code base.
  2. OSS provided clients with ownership over their data. This is a subtle but very significant change. If the source of the program is open, then the data is per definition as well (even if you only use the source to dump the data in a different format).
  3. In a nutshell, OSS brought balance to the force by redistributing solution ownership/risk to the clients who were interested and capable to take it.

That was great for libertarian ideas. But the world has moved on.

Cloud computing suddenly made OSS obsolete (well, from the average clients' point of view). Compare Google Apps with Microsoft's online Office. They both provide the same functionality and we don't know which portions of each product is really OSS, because we don't come directly into contact with the code. We see only the service.

Which brings me to my discussion in this article. In a nutshell, what must Online Services offer the client to provide the equivalent benefits that OSS brought clients in the local execution model?

Firstly, why does this matter? Why should you be concerned about this?
  1. Imagine you build your business on Google Apps and at some point, you are not happy with their security/costs/whatever and want to move on. If you cannot get your data in a reasonable way out of the system, then you are locked into them.
  2. Imagine you suddenly find your sensitive company details in Google's search engine. Somewhere in the EULA you agreed to have your data accessed and now you don't like it anymore (but it is too late).
  3. Imagine Google kills off all the competitors with their "free" service, and then suddenly after all the competition is dead, they start charging exorbitant fees.
Open Service Agreement

I believe online vendors should subscribe to the Open Service Agreement and it should contain the following provisions.


Privacy
All services comes in one of three formats (Service providers to indicate their position here):
  1. Encrypted storage means your data is encrypted with a key that only you have and therefore it is truly secure, irrespective of the domain where the data is located. The Provider must state the encryption algorithms and which implementation of the algorithms they use (so that you know if/when your data is vulnerable to an exploit).
  2. Confidential storage is a best endeavor effort by the service provider. So data is not encrypted but all things being equal, will be kept safely, securely and isolated from other customer's data on the Providers' online storage.
  3. Confidential storage with service analysis where the provider informs you he/she will access your data for value-added-provisions, after all personal bias has been removed. That means, the provider can access trends in your data for their own purposes, but under no circumstances, can they allow the data to be connected to you. This is a provision that is very specific to the data being stored because for instance, if the service provider publishes a trend that indicates that the only person who owns a Xoloitzcuintil dog in Amsterdam also has aids, then it is relatively easy for your neighbors to conclude that it might be you if you are the owner of such a dog (based on the rarity of the breed).
  4. The deletion is final clause states that a provider must provide assurances that data is completely deleted when you delete it. Delete means it is gone and cannot be retrieved for whatever reason at all.
  5. Protection from invasion requires the provider to state their legal position on enforcing the privacy of your data. That means that they have to state in which legal domain/country the data is stored, what the local laws are regarding the state and other people accessing your data and the providers' position on protecting your privacy.

Continuity
Continuity addresses concerns regarding your business continuity when you want to move away from the provider or if the provider stops offering the service.
  1. The provider shall provide Export formats in which you can download all your data in some open format. Open format in this case, means a clearly defined and open source supported format such as XML for which tools exist to manipulate the data.
  2. The provider shall provide an efficient export solution. Efficiency means that you can download your data in a reasonable time and with reasonable effort.
  3. "Living will" code escrow is a provision in which the provider keeps the source code to the product closed, but ensures you that in the case of their demise, that the source code will be available from an escrow scheme under an appropriate license for you to transition your business to another product. Providers should state that they either provide this or not.
Service Level Agreement
SLA's have been around for a long time in the big IT industry, but the relevance of SLA's are even more profound in the cloud computing services model.
  1. Minimum downtime is a clause to provide assurances as to the minimum downtime guaranteed for the service.
  2. Service history publication is a statement from the Service Provider where they agree to publish their performance history so that clients can decide for themselves whether the Provider is compliant with their own SLA claims.
  3. Transaction performance guarantee is a statement as to what the distribution of maximum round trip times will be for the application. This is an indication of how long a client can expect transactions to take on average, with an indication of the worst case scenario.

Free (as in beer)
What does it mean when a service is free and what are the responsibilities of the client and provider to each other in this case?
  1. Life time utility means the product stays free for the lifetime of the clients' need unless stated otherwise. This might be a very long time and if the provider wants to bug out of this responsibility, they need to comply with the termination requirements.
  2. The bug fix clause requires a clear statement whether bugs will or will not be fixed during this period of time.
  3. The provider may "fork" (forking clause) the services using the same OSS or COTS produce code base and provide a commercial alternative to the free service, as long as they stick to their provisions stated in the two points above.
  4. The "What you see is what you get" provision means that the provider is not compelled to include any new functionality in the free product.
Termination
In regards to termination of a service, the provider must clearly state up front the following conditions:
  1. On data access and continuity. For how long and in what format will the data be made available for clients to transition.
  2. On service access and continuity. For how long will the service be made available for clients to transition from before termination.
  3. The performance clause is a guarantee that the service will comply to some reasonable performance during termination (it is difficult to believe that during wind-up, a provider will invest in performance requirements, but if a provider states this up front, then it means that they at least thought about this)
Conclusion
The Open Services Agreement is a suggestion to level the playing field by formalising service risk in a comparable manner and allowing clients to compare services to each other so that they can select the most appropriate service for their needs.

Tuesday, November 03, 2009

Managing vendor risk

I recently came across this post on dirty tricks that vendors use to sell enterprise software (Article). Having worked for American vendors before, I had a chuckle at the obviousness of these tricks, but yes, they happen all the time (I have been lucky and in general worked with very good sales people who didn't do these kinda things).

What lacked in the article was how to manage these risks and so here is my take on managing vendor risk.

Before I say anything about how I think these risks should be managed, just a few things about why I think they are risks.

  1. Humans are bad at estimating risk intuitively. We are not capable of dealing with very small or large numbers in a repeatable, succinct or comparable way.
  2. Humans are all human and sales guys....well, they are sales guys. Successful sales people are very adept at controlling the sales process to their advantage and often that does not include managing the clients' implementation risk.
  3. Not all companies recognise the purchasing of enterprise software as a long and challenging process (not at all like buying a book from Amazon that is done in 30 seconds).


So, how do you reduce the risk of an enterprise implementation?

Firstly, recognise that there is risk. That means literally that you are about to purchase a product at an estimated cost, with a perceived functionality and to be delivered on an agreed upon date. These magical 3 objectives are very hard to disconnect. As a matter of fact, it is nearly always required that one of the three requirements must be decoupled in order to achieve the other two. However, with software product selection, your job is to manage this process in order to optimise that objective (get as much required functionality, as bug free, and as on-time as possible, without cost overruns).

How to manage vendor risk
  1. Get a project plan. You need to read up on project management (not a whole lot) but a side effect of a project plan is that it projects agreements into the future (normally it is seen as a tool to record the past and manage the present). This is a first step to getting the vendor to formally agree how the future is going to look for you during this purchasing process. Amazing how so many of those really good looking candidates don't look so good any more after asking them to commit to the basic project plan.

  2. Write everything down. Normally as part of the project plan, all agreements and critical statements by the vendor must be recorded in writing and signed off by both parties (well, an email to each other stating the obvious is mostly enough). This is the easiest way to prevent misunderstandings and prevent accidental scope creep.

  3. Do a sensitivity analysis. What happens if the products are delivered a bit late or if the functionality promised slips from version 1.x to 1.y? In complex implementations you have conditional dependencies (think of flying people to remote locations to install equipment that has to be delivered independently). A sensitivity analysis not only points out potential huge losses but can also save you money by helping you make optimal decisions up front because it basically gives you a confidence rating in your future plans and you can now do contingency planning.

  4. Quantify your risks if possible. Quantification is hard to do well because you need a lot of input data (that is difficult and specialised to come by) but when you are dealing with a complex project, then lack of quantification means you are trying to compare high risks with high risks and which one is higher? Quantification is typically a cardinal indication of risk that can be compared to alternatives.

  5. Control the sales process. This is something that vendors specialise in taking over from you and you have to take this back. Remember, they spend all their time developing strategies on how to control the sales process while you only buy this product once every X years (or so you hope). If you want to evaluate the product, work with the vendors to arrive at acceptable evaluation criteria and never accept the vendors' claims of compliance. Testing the product does have downsides (often it is like driving a sports car without any training) and many vendors today prefer you to consult with reference accounts. That is fine, but make sure that you do your homework on the reference you visit or speak to. Who are they and how comparable is their situation to yours? Also, be careful that it is not just a vendor trick by providing a false reference.

  6. Put the responsibility of the success of the project on the vendor. By splitting the sale into a "fixed price" product component and a "time and materials" implementation component, the vendor effectively makes you take some implementation risk on yourself. Hey, the vendor implements this solution daily (it is their job/product) so they should be a in a position to fix the implementation costs of the product and include it as a fixed price component in the deal. That way, they only get paid if the product does what it is supposed to do and they will only get paid what was agreed up front.

  7. Agree on a change control policy. "Scope creep" is a typical phenomenon that vendors/implementers use to ratchet up the cost of projects. Turn this agreement to your advantage. If they can charge you for adding features to the sale, you can deduct costs for features that don't work as agreed.

  8. What about vendor lock-in? This is a difficult challenge to solve in a generic way because the real objective here is to minimise your long term risk or total cost of ownership of the functionality. This can be achieved by buying open or closed source systems (both have their advantages or disadvantages) but here are my 2 cents on how to navigate this minefield. Guard your gates by making a "living will". Normally, it is the data that is important in systems and not the system itself, so if you can get your data easily in and out of the system (and think of transforming the data in between transfer in order to fit the new system) you already protect yourself from vendor lock-in. So a "living will" is a way for you to plan the recovery from possible systemic failure into the procurement of the product in the first place.
  9. Lastly, get your whole team on-board. Large companies are fraught with internal politics and failure to notify and get agreement from all relevant stakeholders means that you could be fighting your own company during the implementation. Internal resistance can easily allow the vendor off the hook on critical issues because they can claim obstruction and therefore, insist on change control.

Wednesday, July 22, 2009

Wicket renaming your submit button


add(new Button("submit", new ResourceModel("submit")));

Saturday, July 04, 2009

ATI HD 4850 Linux issues

I recently bought a Dell XPS with an ATI HD4850 graphics card in it. After struggling with this card for weeks, this is where I am now (the reason why I wrote this is to warn other Linux users about this card).

Config: Ubuntu Karma with Dual head display (HP Digital screen and Phillips analogue screen)
Driver: best performance so far is the AMD catalyst 9.6 driver.


Issues:
1. I can now enable xinerama or big desktop from within the driver amdcccle interface.
2. However, with big desktop, the card cannot put the screen into power savings mode (sleep) and on trying, the big desktop settings are lost and you have to restart X.
3. With xinerama, all looks ok, the screens can sleep, but on waking up, something crashed in the card and none of the 3D functions work (including screensavers etc.). You cannot even run xglfxinfo (it just silently hangs). This again means it is only a question of time before you have to restart X.
4. The open source drivers just crash when X starts.

UPDATE: The latest Open Source drivers in Ubuntu 9.1 works well with the only issue being the top resolution that you can get being a bit lower that what is possible with the closed source drivers.

Sunday, June 07, 2009

Wicket stuff I always forget

This is the stuff that I always need, but don't use enough to remember.

How to retrieve localized text programatically in a wicket application?
item.add(new Label("status",new Localizer().getString(session.getStatus().toString(), new AuthHomePage())));


or


public class MyChoiceRenderer extends ChoiceRenderer {

@Override
public Object getDisplayValue(Object object) {
return new Localizer().getString(object.toString(), new AuthHomePage());
}

@Override
public String getIdValue(Object object, int index) {
return object.toString();
}

}




How to use a checkbox to set a specific value (not boolean) in a wicket form:

private class InvoiceableCheckBoxModel extends Model{
private IModel wrappedmodel;


public InvoiceableCheckBoxModel(IModel model){
this.wrappedmodel=model;

}

@Override
public Object getObject(){
return Boolean.FALSE;
}

@Override
public void setObject(Object object){
if (object.equals(Boolean.TRUE)){
wrappedmodel.setObject(SessionStatus.invoice);
}
}
}


How to turn an array into a List?

Arrays.asList(SessionStatus.values())

Monday, April 20, 2009

eclipse vs. netbeans with maven, wicket and db4o

Ok, I am not going to go into great detail why, but the bottom line is that I reached a point where I decided to move over from Eclipse to Netbeans.

This turned out to be slightly challenging because:
  1. I used mvn jetty:run as a webserver which is simple/embedded and just works. However, netbeans doesn't automatically build projects when your source changes, you need to tell it to track changes to your sources. Here is the config for the webapp pom to track any changes to your source.


  2. <plugin>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>maven-jetty-plugin</artifactId>
    <configuration>
    <scanIntervalSeconds>5</scanIntervalSeconds>
    <contextPath>/practice</contextPath>
    <scanTargetPatterns>
    <scanTargetPattern>
    <directory>src/main/java</directory>
    <includes>
    <include>**/*</include>
    </includes>
    </scanTargetPattern>
    </scanTargetPatterns>
    </configuration>
    </plugin>

  3. A second problem was how to reference the sources of my war and use it in a second project in a jar project. netbeans give dependency management completely over to maven so you cannot point one project at another at all. You have to do this in maven. What this means is that your project must successfully build in maven before you even think of making it work in netbeans. The strategy to achieve this requires you:

    1. Create a high level project (parent.pom) and include your inter dependent projects as modules. Maven will work out which ones need to be built first. You then need to set the correct dependency in the dependent project to basically look for the compiled classes of the webepp in a special jar in your local maven repository (wait for it the trick will come in a second).

      <dependency>
      <groupId>com.musmato</groupId>
      <artifactId>webapp.war</artifactId>
      <type>jar</type>
      <classifier>classes</classifier>
      <scope>provided</scope>
      <version>1.0</version>
      </dependency>
      Note a few things. Scope is provided (jar is in your local maven repo). We use a classifier which is slightly unusual (see link as to why)

    2. You now need to somehow get maven to build and package your war sources in a jar. here is an extract from my pom. That is it. Here is info on the maven war plugin

      <plugin>
      <artifactId>maven-war-plugin</artifactId>
      <version>2.1-beta-1</version>
      <configuration>
      <attachClasses>true</attachClasses>
      </configuration>
      </plugin>

      This bit of magic will package your classes in a separate war and ensure that it is stuck in your local maven repo.


  4. Once you have this all in place, running
    mvn clean install
    on the parent project will build your dependencies and install them for each other to use



So, in a nutshell, you cannot reference a war file without using the maven war plugin and storing the classes seperately.