8. Filter and Project

We have quietly used filter and project in most of our examples, without specifically highlighting them. Let's take a minute to do so. Simply put, the Filter operation is the where clause specification in a StreamInsight LINQ query, along with the select clause that outputs the filtered events. Consider the query [TopK]. Suppose we wanted to see the output only for TollStation 1. The time plot in Figure 19 shows the effect of this operation on the output of the [TopK] query.

image

Figure 19. Filter for toll booth 1 applied to the result of [TopK] query

The query becomes:

var query = from e in topK
               where e.TollId == "1"
               select e;

Click here to view code as image

The filter clause here is e.TollId == "1". This can be any logical expression allowed by LINQ. UDFs (User Defined Functions, which we will see shortly) find their common use here.

A Project operation “transforms and projects out” the payload fields for the stream output, packaged into a known type (as we have seen in examples so far) or into an anonymous type (when used for query composition) as part of the select clause. Projection allows you to transform the payload – in terms of in-place changes to a field, or in adding or removing fields. Similar to the case for filters, there are no limitations on the kinds of LINQ and C# expressions that can be supported for projection transforms.