Home Up



Filtering Knowledge

  By Roy D. Follendore III

Copyright 2002 Roy D. Follendore III

For the purposes of this paper, it should be understood that there is a subtle functional differences between the process of searching for knowledge and the activity of filtering knowledge.  

Searching is an active process.  It implies there is something known in advance that is considered important. There is a unique, specific and individual act of identification matching. An individual act of separation and isolation may or may not be performed after identification.  Search is a process leading up to an explicit act of identification.  For instance, a search for course sand does not necessarily imply that we will separate the course sand from the fine sand.  There is a specific kind of search that is the result or product of isolation.  

Filtering takes place in combination with a search being made so that it produces a product after isolation.   Within a filtering process there may be explicit or broad binary kinds of optimal isolation taking place.  Filtering therefore involves searching directly through the isolation process.  Isolation is always a result of filtering.  As a result, what is left behind may be more optimum than that which has been filtered.  The opposite can also be true.  What is passed through may be more optimum than what tis left behind.  We can also reverse the idea and say that what is left is less optimum than what was filtered.  It all depends on your perspective and objective.  The objective of filtering can therefore be to isolate optimal or the less optimal, a secondary result of filtering is the concept of search.  Because filtering can isolate broadly it can be used as a means of reducing the complexity and error rate of search processes.

The difference between the two forms of discrimination involve the result which can be easily seen with a physical demonstration.  If a box of uniform square blocks and uniform round balls were randomly mixed together on a physical surface we could use a computer to discriminate an individual block from the balls and blocks.  The activity of doing this would be a search.  If there were 100 blocks and 100 balls such a discrimination might take 1 search or 100 searches before being successful.  Once such a search has been successfully accomplished, there would be no change in the relationships of the balls and blocks.  If the surface were constructed so that it was a template it would allow a particular shape, round or square to fall through.  The result is that the blocks and balls can be isolated and thereby discriminated.  As a rule, filtering always physically disturbs the relative relationship of components because the template is physically displacing units under test.  A search does not require this because individual attributes of each unit under test is being matched. The attributes of filtering are not required to be known for filtering however.  If in our example, knowledge of our template discrimination process is unknown then a complete assumption of search through filtering can not be made.  However, even if filter process of balls and blocks are lost then search overhead is limited to two uniform groups which reduces the maximum number of search transactions to two. 

It is easy to conceptually misrepresent the concept of searching activities in terms of filtering because both are ultimately discrimination transactions.  If the results happen to be the same, then the means do not seem important.  The importance of these subtle differences is this.  With search, the attributes of the search criteria must be known.  With filtering, the template for discrimination can be autonomous and unknown.  Furthermore, intelligent use of filtering can be used to both reduce search error and improve search performance.  Filtering use of intelligent templates can be associated with the units under test to be inherently contextual and secure. It is possible to do secure blind discrimination, a thing that is impossible to accomplish with search because the search criteria is effectively the search.    

The Filtering of Contextual Wisdom

If you have been reading about the concept of Noise to Knowledge on www.noisetoknowledge.com you should be aware that all knowledge can be said to be comprised of elements systematically arising from noise.  Knowledge is complex because it involves contextual arrangements of elements.  The context of knowledge is relative to a  very broad concept defined as wisdom.  Wisdom, as it is being used here, is much more than a passive concept.  Wisdom implies choice.  Wisdom is the contextual template for choices in filtering.  Without a basis for making choices there can be no filtering.  Where Noise to Knowledge represents the process of context, wisdom represents the applied purpose of context.  Change the intended purpose and the process changes.

The purpose of knowledge context is typically lost during the process of contextual creation. We could diagram each sentence and evaluate the verbs, adjectives, nouns and pronouns to get at the context.  That could then infer the reason (s) why the sentence was created.  We could then aggregate the reasons for sentences to justify the purpose of the paragraph context.  This could then be interpreted further to justify the reasons for the story.  The problem with this is that such a procedure implies that the intent is "rationally" embedded within the context of Noise to Knowledge.  But wisdom arising from knowledge is not necessarily rationally linear. Satire, analogies, and scenarios are just some of the nonlinear knowledge that can be wisely used.  Human communication is rich in contextual justifications.  Any attempt to justify wisdom context from structure after the fact of authorship is doomed to failure before it begins.  

Filtering Knowledge Through The Objective Story

Filtering knowledge requires a template through which rational relationships can be explicitly separated.  In order to do this it is necessary to introduce a process by which this can be accomplished.  It is easy to understand why a filtering template would somehow need to be be mapped to the context of the knowledge.  The physical analogy we can make to filtering knowledge would be that of screening of sand.  Each particle of sand is unique while the screen is uniform.  The size of the screen we choose would dictate the product of screening.  If the screen size is too small, there may not be any product which gets through and as a consequence too little granular separation.  If the screen is too large, there is still too little separation.  Unlike an explicit search for a specific sand particle size, the screening process is generalized.  It is the redundancy of the process to the grains of sand as a whole that makes filtering so productive over explicit search methodologies. 

Is it possible to filter knowledge in a similar manner so that we can achieve this kind of efficiency?  The answer is yes, but this is where the analogy of screening sand breaks down.  The problem with filtering knowledge is that knowledge is more complex than filtering sand.  For one thing, knowledge does not have the physical dimension of sand.  As a consequence, this means that we are talking about is the use of contextual logical and rational dimensions as criteria for knowledge filtering.  To accomplish this would require the establishment of integral meta dimensions which are universal to knowledge content.    

Suppose we wanted authors of knowledge to include a meta level rational context.  To do this we first would need understand this concept of rational context.  Rather than mathematical rationality, rational context may be defined with respect to this over riding idea of wisdom. Rational context is really the objective story that is usually told indirectly.  This is the objective story that is implied.  There is always an "objective story" within which knowledge has been authored.  This means that there is authored intent implied within this story.  Moreover, depending on the how knowledge is adopted within a given organization, there can also be more than one rational story implied. The personal story of the writer can be different from the story presented by the organization. 

Missed Context

The context of Noise to Knowledge as we are being served it is missing from authored communication.  How we might choose to tag a rational story depends on what we might want to use the knowledge. The tags are selected to tell the story.  Because of this, knowledge can  be thought of as being raw or processed.  Processed knowledge have meta level external tags.  Raw knowledge have no external tags.  Processing knowledge to produce meta tag stories can become costly if it is not handled as part of the authoring process.  The reason is that the analysis of knowledge after the fact implies the recovery of a story that has been previously discarded and lost.  It can be difficult to impossible to recover the meta story related to the authors intent. The recovery of the meta organizational intent should not be considered the same thing.

Granular Filtering of Noise to Knowledge

The effective granularity of Noise to Knowledge is also an idea that must be better understood if we are to understand the concept of knowledge filtering.  Every element of knowledge, including the knowledge itself may have a story, even though in practice they may not be needed.   The disadvantage of meta tagging is primarily the overhead involved. Someone needs to initially create as well as manage the meta stories over time.  The advantage of tagging every level means that the knowledge becomes more rationally adaptable to filtering.  Meta tagging stories provide hooks that allow the components of knowledge to be broken down and rearranged.  Since rearrangement components of knowledge can essentially create new knowledge, meta tags become the rational means by which the creation of new knowledge can be accomplished.   Because we have related knowledge context we can relate knowledge to context and thereby create new knowledge.  Filtering is the very first step in being able do this.





Copyright (c) 2001-2007 RDFollendoreIII All Rights Reserved