Relating Stars to Human Knowledge

Last night I was out for a workshop and there post dinner we all were interacting and sharing our ideas and thoughts what interests us. After some time for some reason I separated myself out of the crowd and was back into the thought process which I like to get into. I was looking at the sky and then all I could see was stars. Some bright and some not so bright. This triggered a thought as do they also represent the way knowledge is stored in human brain? This post relates back to the previous post where I put my thoughts about a Self Learning Systems. In this post we will discuss about how the knowledge can be accumulated over a period of time.

The accessibility of any piece of information in our brain has a factor of when it happened or when in past we came across the incident. Like the stars in the sky, there are few bright ones and few not so bright ones. Each star appears to us with a different intensity and brightness. The one which appears brightest among them are the ones who are closer to us. The classic example is Sun. Same way the things which we remember easily and better are the events which happened in recent past and as they day passess by our memories of the same also starts to fade.

Another similarity of Knowledge and Stars is their lifecycle. The stars are born, they grow to their peak capacity and then they die and become a dwarf star or black hole. The similar things happen to our knowledge and learning as well. We Learn and the knowledge stays with us for a while and after sometime it fades away (if we don't use it) and vanishes. But as it happens with stars they get transformed into a dwarf or black hole, the knowledge we accumulate moves to a non-accessible or not so easily accessible location in our brain. But it never vanishes. It remains there.

The advantage of using a computer system here is that we have access to all those details uniformly no matter how old they are and when they were used last. In the last post of mine when I talked about the Self Learning System, I mentioned about the able to learn from user input. The system must ensure that the knowledge, data, query result etc are accessible with a reference to when was that acquired and when was it used last (with an option to view the log as how many times the same piece of knowledge was used).

How can this be made possible? This is the million dollar question one will ask? Well to answer that we need the following:

1. A computer program to capture the User input Query.
2. A Service to query and retrieve all the different possible ways to solve the problem from available vendors or service providers.
3. A mechanism to store the query criteria, output and the reasoning used to arrive at the result for future purposes.
4. A mechanism to log the actions in the system and show them to the user when the need arises.

In next few posts I will attempt to draw an architecture of how this kind of system may look like.

Until Next Time... :)

Self Learning Systems

It was yesterday around lunch time when I went out to buy myself lunch I got this thought. Been working in Software Development for more than 11 years now, I sometime get this feel that computers are nothing more than a dumb orderly. I still remember when someone (don't remember the name) explained me the Expansion of COMPUTER (Completely Obedient Machine Provided User Types Everything Right) first time. Not sure how correct this expansion is but it sounds very much correct in context of computers.

Coming back to the idea what I had. I was thinking if we can put in a place a mechanism to store the last user response to a certain user query; I am sure we turn this dumb machine to something more intelligent. Say for example I query for Pizza Order for Morningside, Brisbane with delivery option available after 10pm. There are different pizza shops in the same suburb. They will publish their services in regard to which suburb they can deliver, if they have a time limit for delivery etc. First time my computer does not know about which vendor to choose. But say I select a pizza vendor I know is good and place next request to get the menu from the vendor. After selecting the Pizza from the menu I place the order and pay by my visa card. Next time if I place the same request it should be able to query the vendor and get the menu in front of me from the vendor. At this point the computer the intelligent system will be able to tell me that last time I ordered the Chicken Pizza and paid by my visa card. Whether I would like to continue with the same or want a different pizza or want to pay by cash or some other card.

Where I am coming from is. The system in this context is not just able to obey my order but is also able to suggest me based on the previous decision I made. So next time on the system is exhibiting the intelligence and knows my preference as well.

In order to build such a system we need:

1. Client Agent which will store the user query and also the results along with a validity (if applicable).
2. Server Agent with whom the client will be interacting and passing on the query for which the client agent does not have the answer.
3. Global Service Database where every vendor's services are listed.
4. Query Server to perform query based on the parameters supplied. Also queries should be based on the meaning. Not just the attributes. eg. If I am after the pizza shop in Brisbane, Australia it should not bring back the results for Brisbane, California.
5. The queries should be based on Ontology so that more meaning can be added to the Search criteria.
   
6. The services published also have some sort of Ontological Commitment so that it can be retrieved by the Query server.
   

If we have similar kind of infrastructure in place. I am sure we can achieve an intelligent computer (and a computer network in-turn) and realize the vision of Semantic Web.

Until Next Time...:)

Developing Intelligent Applications on Web

I was reading a book on Creating meaning on Web and sharing information on Semantic Web. While going thru pages I came across one good extract from an article from Fensel and Brodie, 2003. They mentioned a 3 level solution to the problem of developing intelligent applications on the web.

Information Extraction: In order to provide access to information resources, information extraction techniques have to be applied providing wrapping technology for a uniform access to information.

This directly corresponds to different approaches which is taken to access the information and interpret them.

Processable Semantics: Formal languages have to be developed that are able to capture information structures as well as meta-information about the nature of information and the conceptual structure underlying an information source.

This here refers to the recently developed specifications like XML, RDF, OWL in conjunction with Syntactical and structural approaches.

Ontologies: The information sources have to be enriched with semantic information using the language mentioned in step above. This semantic information has to be on a vocabulary that reflects a consensual and formal specification to the conceptualization of the domain, also called an ontology.

This directly corresponds to the recent efforts made by scientific community to enrich the Semantic Web in terms of Meta Annotation and Term Definitions.

While much has been talked about the later two. The first element i.e. the information extraction is still a puzzle for many people in the scientific community. This is due to the heterogeneous nature of the web. Where the heterogeneous not only in terms of data, but also in terms of the logic to interpret the data and the context in which the data element is being used.

To retrieve the information semantically we need to build the data in semantic way. Unless we adopt to a standard which is sufficient enough to describe the smallest entity in the system and can be scaled to the largest entity using the same mechanism, the achieving true semantic is almost next to impossible. In one of my earlier post about Atomicity I had explained how the principle of atomicity can be scaled from an atom to a large scale enterprise database. May be we can leverage on the same concept and build something which is good enough for the Semantic Web.

Until Next Time...:)

Information System Design

Lately I've been busy acquiring in-depth knowledge of Web Services and why do we need them. In this journey I came across few good things and few not so good things. My primary concern was always the design issues which forces one to use Web Services and on the top of that the architectural concerns, constraints etc. Not to forget the Business Requirement which adds to the constraints we already have to deal with.

In this process I came across a very good which described about the Basics of an Information System design. Lets first get our head around what an Information System consists of. In a Typical information system we have:

1. Client Layer: The front-face of the application as seen by the client. This is typically a HTML page rendered by the web browser.
2. Presentation Layer: The logic which is used to render the data and build the presentation for the client. The best example of this is XSL rendering which is being used in eBay web pages.
3. Business Logic Layer: Set of programs which process data entered by user as per the business rules defined in the system and then pass on to data access for storing. When retrieved the raw data from the data access layer these programs are responsible for converting them into a format which is meaningful to the user.
   
4. The Data Access Layer: The data access layer manages the way data is stored in the system. It exercises the Normalization / De-Normalization principle, Establishes relationship among data elements before persisting them into the data store. This understands the format in which the data is being stored.
   
5. The Resource Layer: This is the core of an information system. The data which is stored in Database, File Systems, Active Directory etc form this layer of the information system.
   

Having known the different parts of an information system we then go for designing how the information will flow and how the different parts of the system will interact with each other. There are few different approaches which is put into practice by architects and designers around the world.

Top Down Approach:
This is the most common approach which is followed by the architects and designers around the world. In this methodology the Functionality or the Business Requirement drive the system design and their components (abstraction) etc. Here client is the center of focus always. The system is designed to keep client in mind, how will they interact with the system etc. The name Top-Down approach comes from the way the design of the system evolves. The designer first sees how the client is going to use the system ie Presenation Layer, then they design the Business Logic or Application Logic components of the same and then at the end the resources (data store) is finalized.

The steps involved in Top Down approach is:

1. Define access channels and client platforms
2. Define presentation formats and protocols for the selected clients and protocols
3. Define the functionality necessary to deliver the contents and formats needed at the presentation layer.
4. Define the data sources and data organization needed to implement the application logic.

This approach of design has few advantages like it emphasizes the final goal of the system, and at any point in time it can cater to changes in both functional and non-functional needs of the system. But on the other side it can be applied to only the system which is developed from the scratch.

Bottoms Up Approach:
The Bottoms Up approach of system design is not a preferred choice of the designer to start with. It occurs out of necessity than the choice. There are systems which exists in this world serving their original purpose with great accuracy and the organizations rely heavily onto it for their day to day business function. But these systems have become legacy system as they are not able to cater the needs of requirements than they are originally designed for. These systems have a fixed set of functionality defined and the development effort to make changes in the system to cater the new requirements are not viable option for the organization. Under such circumstances the Bottoms Up approach is used.

The Steps involved in Bottoms Up approach is:

1. Client Defines access channel and Client Platforms.
2. Examine legacy system and any other resources and the functionality they offer.
3. Build a wrapper around the existing resources and also build an adapter to provide a consistent interface to the outside world.
4. Build adapters around the application logic so that it can be used by any client and access channels and protocols.

As stated earlier this design approach is not by choice. Rather it is dedicated by the need to integrate the services provided by legacy systems. This is one area where much of spending in today's world is happening. Most of the buzzwords today like Web Services, SOA, EAI all are primarily targeting in this area.

Until Next Time...:)