Rectangle 27 20

I have written an open source project, Dynamic Expresso, that can convert text expression written using a C# syntax into delegates (or expression tree). Text expressions are parsed and transformed into Expression Trees without using compilation or reflection.

You can write something like:

var interpreter = new Interpreter()
                .SetVariable("service", new ServiceExample());

string expression = "x > 4 ? service.aMethod() : service.AnotherMethod()";

Lambda parsedExpression = interpreter.Parse(expression, 
                        new Parameter("x", typeof(int)));

parsedExpression.Invoke(5);

reflection - How can I evaluate a C# expression dynamically? - Stack O...

c# reflection eval
Rectangle 27 3

That simply isn't supported by the current C# compiler, and I haven't heard about any changes in vNext. Of course, strictly speaking it isn't defined for C# 3 / 4 - there is just a "is defined elsewhere" (actually, AFAIK: the spec for handling expression tree construction still isn't formally documented; this could be a positive thing, as it is hard to argue that it will require specification changes ;p).

The funny thing is: from .NET 4.0 onwards, the expression tree API does support mutate (in this case, see Expression.Increment and Expression.PostIncrementAssign) - so you could create the expression tree at runtime via Expression.* code, but frankly that is a pain and hard to manage. So there is potential for this to change, but don't be too hopeful.

Also keep in mind - the expression tree analysis to pull it back out again is far from trivial. Doable, sure; easy: no.

Yeah, we never did ship that spec, sorry. Regarding the expression tree API: we considered it, along with considering supporting conversions from statement lambdas to expression trees. The thing is, we do not have a compelling end-user scenario that is so great that it would cause us to cancel or delay more interesting work (like dynamic, or async). It would be nice to fill in those holes, but "nice" is insufficiently compelling.

Good to know, tnx Mark and Eric.

Thanks Marc - agreed, the second step is non-trivial, but certainly not harder than implementing a linq provider. I simply wanted something lighter weight than Linq to hide that complexity from the end users of my API and provide them with query format that was agnostic to the underlying storage provider. It looked so elegant that I was SURE it would work! Oh well.

@Eric keep shipping other awesome stuff and I'll forgive you :p agree that this spec has a tiny audience - writing new tools is better. Finding an awesome example for the statement scenario is also elusive, agreed.

c# - Creating expression trees from lambda - Stack Overflow

c# expression-trees
Rectangle 27 7

You're correct in noting this is not a trivial problem.

The classical way that compilers handle it is a Directed Acyclic Graph (DAG) representation of the expression. The DAG is built in the same manner as the abstract syntax tree (and can be built by traversing the AST - perhaps a job for the expression visitor; I don't know much of C# libraries), except that a dictionary of previously emitted subgraphs is maintained. Before generating any given node type with given children, the dictionary is consulted to see if one already exists. Only if this check fails is a new one created, then added to the dictionary.

Since now a node may descend from multiple parents, the result is a DAG.

Then the DAG is traversed depth first to generate code. Since common sub-expressions are now represented by a single node, the value is only computed once and stored in a temp for other expressions emitted later in the code generation to use. If the original code contains assignments, this phase gets complicated. Since your trees are side-effect free, the DAG ought to be the most straightforward way to solve your problem.

As I recall, the coverage of DAGs in the Dragon book is particularly nice.

As others have noted, if your trees will ultimately be compiled by an existing compiler, it's kind of futile to redo what's already there.

I had some Java code laying around from a student project (I teach) so hacked up a little example of how this works. It's too long to post, but see the Gist here.

Running it on your input prints the DAG below. The numbers in parens are (unique id, DAG parent count). The parent count is needed to decide when to compute the local temp variables and when to just use the expression for a node.

Binary OR (27,1)
  lhs:
    Binary OR (19,1)
      lhs:
        Binary GREATER (9,1)
          lhs:
            Binary ADD (7,2)
              lhs:
                Binary MULTIPLY (3,2)
                  lhs:
                    Id 'Bar' (1,1)
                  rhs:
                    Number 5 (2,1)
              rhs:
                Binary MULTIPLY (6,1)
                  lhs:
                    Id 'Baz' (4,1)
                  rhs:
                    Number 2 (5,1)
          rhs:
            Number 7 (8,1)
      rhs:
        Binary LESS (18,1)
          lhs:
            ref to Binary ADD (7,2)
          rhs:
            Number 3 (17,2)
  rhs:
    Binary EQUALS (26,1)
      lhs:
        Binary ADD (24,1)
          lhs:
            ref to Binary MULTIPLY (3,2)
          rhs:
            ref to Number 3 (17,2)
      rhs:
        Id 'Xyz' (25,1)
t3 = (Bar) * (5);
t7 = (t3) + ((Baz) * (2));
return (((t7) > (7)) || ((t7) < (3))) || (((t3) + (3)) == (Xyz));

You can see that the temp var numbers correspond to DAG nodes. You could make the code generator more complex to get rid of the unnecessary parentheses, but I'll leave that for others.

Thanks very much! Will take a look.

c# - Efficiently eliminate common sub-expressions in .NET Expression T...

c# .net algorithm optimization expression-trees
Rectangle 27 84

Ooh, it's not easy at all. Func<T> represents a generic delegate and not an expression. If there's any way you could do so (due to optimizations and other things done by the compiler, some data might be thrown away, so it might be impossible to get the original expression back), it'd be disassembling the IL on the fly and inferring the expression (which is by no means easy). Treating lambda expressions as data (Expression<Func<T>>) is a magic done by the compiler (basically the compiler builds an expression tree in code instead of compiling it to IL).

This is why languages that push lambdas to the extreme (like Lisp) are often easier to implement as interpreters. In those languages, code and data are essentially the same thing (even at run time), but our chip cannot understand that form of code, so we have to emulate such a machine by building an interpreter on top of it that understands it (the choice made by Lisp like languages) or sacrificing the power (code will no longer be exactly equal to data) to some extent (the choice made by C#). In C#, the compiler gives the illusion of treating code as data by allowing lambdas to be interpreted as code (Func<T>) and data (Expression<Func<T>>) at compile time.

Lisp doesn't have to be interpreted, it can easily be compiled. Macros would have to be expanded at compile time, and if you want to support eval you would need to start up the compiler, but other than that, there's no problem at all doing that.

"Expression<Func<T>> DangerousExpression = () => dangerousCall();" is not easy?

@mheyman That would create new Expression about your wrapper action, but it would have no expression tree info about internals of dangerousCall delegate.

c# - converting a .net Func to a .net Expression> - Stack O...

c# .net lambda expression func
Rectangle 27 247

Lambda expressions are a simpler syntax for anonymous delegates and can be used everywhere an anonymous delegate can be used. However, the opposite is not true; lambda expressions can be converted to expression trees which allows for a lot of the magic like LINQ to SQL.

The following is an example of a LINQ to Objects expression using anonymous delegates then lambda expressions to show how much easier on the eye they are:

// anonymous delegate
var evens = Enumerable
                .Range(1, 100)
                .Where(delegate(int x) { return (x % 2) == 0; })
                .ToList();

// lambda expression
var evens = Enumerable
                .Range(1, 100)
                .Where(x => (x % 2) == 0)
                .ToList();

Lambda expressions and anonymous delegates have an advantage over writing a separate function: they implement closures which can allow you to pass local state to the function without adding parameters to the function or creating one-time-use objects.

Expression trees are a very powerful new feature of C# 3.0 that allow an API to look at the structure of an expression instead of just getting a reference to a method that can be executed. An API just has to make a delegate parameter into an Expression<T> parameter and the compiler will generate an expression tree from a lambda instead of an anonymous delegate:

void Example(Predicate<int> aDelegate);
Example(x => x > 5);
void Example(Expression<Predicate<int>> expressionTree);

The latter will get passed a representation of the abstract syntax tree that describes the expression x > 5. LINQ to SQL relies on this behavior to be able to turn C# expressions in to the SQL expressions desired for filtering / ordering / etc. on the server side.

Without closures you can use static methods as callbacks, but you still have to define those methods in some class, almost certaily increasing the scope of such method beyond intended usage.

FWIW, you can have closures with an anonymous delegate, so you don't strictly need lambdas for that. Lambdas are just eminently more readable than anonymous delegates, without which using Linq would make your eyes bleed.

c# 3.0 - C# Lambda expressions: Why should I use them? - Stack Overflo...

c# c#-3.0 lambda
Rectangle 27 4

Lambdas is not the only way to introduce laziness. Lazy evaluation is relatively straight-forward using Expression Templates in C++. There is no need for keyword lazy and it can be implemented in C++98. Expression trees are already mentions above. Expression templates are poor (but clever) man's expression trees. The trick is to convert the expression into a tree of recursively nested instantiations of the Expr template. The tree is evaluated separately after construction.

The following code implements short-circuited && and || operators for class S as long as it provides logical_and and logical_or free functions and it is convertible to bool. The code is in C++14 but the idea is applicable in C++98 also. See live example.

#include <iostream>

struct S
{
  bool val;

  explicit S(int i) : val(i) {}  
  explicit S(bool b) : val(b) {}

  template <class Expr>
  S (const Expr & expr)
   : val(evaluate(expr).val)
  { }

  template <class Expr>
  S & operator = (const Expr & expr)
  {
    val = evaluate(expr).val;
    return *this;
  }

  explicit operator bool () const 
  {
    return val;
  }
};

S logical_and (const S & lhs, const S & rhs)
{
    std::cout << "&& ";
    return S{lhs.val && rhs.val};
}

S logical_or (const S & lhs, const S & rhs)
{
    std::cout << "|| ";
    return S{lhs.val || rhs.val};
}


const S & evaluate(const S &s) 
{
  return s;
}

template <class Expr>
S evaluate(const Expr & expr) 
{
  return expr.eval();
}

struct And 
{
  template <class LExpr, class RExpr>
  S operator ()(const LExpr & l, const RExpr & r) const
  {
    const S & temp = evaluate(l);
    return temp? logical_and(temp, evaluate(r)) : temp;
  }
};

struct Or 
{
  template <class LExpr, class RExpr>
  S operator ()(const LExpr & l, const RExpr & r) const
  {
    const S & temp = evaluate(l);
    return temp? temp : logical_or(temp, evaluate(r));
  }
};


template <class Op, class LExpr, class RExpr>
struct Expr
{
  Op op;
  const LExpr &lhs;
  const RExpr &rhs;

  Expr(const LExpr& l, const RExpr & r)
   : lhs(l),
     rhs(r)
  {}

  S eval() const 
  {
    return op(lhs, rhs);
  }
};

template <class LExpr>
auto operator && (const LExpr & lhs, const S & rhs)
{
  return Expr<And, LExpr, S> (lhs, rhs);
}

template <class LExpr, class Op, class L, class R>
auto operator && (const LExpr & lhs, const Expr<Op,L,R> & rhs)
{
  return Expr<And, LExpr, Expr<Op,L,R>> (lhs, rhs);
}

template <class LExpr>
auto operator || (const LExpr & lhs, const S & rhs)
{
  return Expr<Or, LExpr, S> (lhs, rhs);
}

template <class LExpr, class Op, class L, class R>
auto operator || (const LExpr & lhs, const Expr<Op,L,R> & rhs)
{
  return Expr<Or, LExpr, Expr<Op,L,R>> (lhs, rhs);
}

std::ostream & operator << (std::ostream & o, const S & s)
{
  o << s.val;
  return o;
}

S and_result(S s1, S s2, S s3)
{
  return s1 && s2 && s3;
}

S or_result(S s1, S s2, S s3)
{
  return s1 || s2 || s3;
}

int main(void) 
{
  for(int i=0; i<= 1; ++i)
    for(int j=0; j<= 1; ++j)
      for(int k=0; k<= 1; ++k)
        std::cout << and_result(S{i}, S{j}, S{k}) << std::endl;

  for(int i=0; i<= 1; ++i)
    for(int j=0; j<= 1; ++j)
      for(int k=0; k<= 1; ++k)
        std::cout << or_result(S{i}, S{j}, S{k}) << std::endl;

  return 0;
}

c++ - Is there actually a reason why overloaded && and || don't short ...

c++ c++11 operator-overloading logical-operators short-circuiting
Rectangle 27 58

Using Expression<T> you are explicitly creating an expression tree - this means that you can deal with the code that makes up the query as if it were data.

The reason for this is that LINQ providers (like LINQ to SQL for example) inspect the query itself to determine the best way to translate the C# expressions into a T-SQL query. Since an expression tree lets you look at the code as data the provider is able to do this.

Don't forget that Expression Trees are not exclusive to LINQ providers anymore - for example, they're one of the main underlying concepts of the Dynamic Language Runtime.

c# - Difference between Expression> and Func<> - Stack Overflow

c# linq
Rectangle 27 80

Operating on the syntax tree

Here is an implementation based on Boost Spirit.

Because Boost Spirit generates recursive descent parsers based on expression templates, honouring the 'idiosyncratic' (sic) precedence rules (as mentioned by others) is quite tedious. Therefore the grammar lacks a certain elegance.

I defined a tree data structure using Boost Variant's recursive variant support, note the definition of expr:

struct op_or  {}; // tag
struct op_and {}; // tag
struct op_xor {}; // tag
struct op_not {}; // tag

typedef std::string var;
template <typename tag> struct binop;
template <typename tag> struct unop;

typedef boost::variant<var, 
        boost::recursive_wrapper<unop <op_not> >, 
        boost::recursive_wrapper<binop<op_and> >,
        boost::recursive_wrapper<binop<op_xor> >,
        boost::recursive_wrapper<binop<op_or> >
        > expr;

The following is the (slightly tedious) grammar definition, as mentioned.

Although I don't consider this grammar optimal, it is quite readable, and we have ourselves a statically compiled parser with strongly typed AST datatype in roughly 50 lines of code. Things could be considerably worse.

template <typename It, typename Skipper = qi::space_type>
    struct parser : qi::grammar<It, expr(), Skipper>
{
    parser() : parser::base_type(expr_)
    {
        using namespace qi;
        expr_  = or_.alias();

        or_  = (xor_ >> "or"  >> xor_) [ _val = phx::construct<binop<op_or >>(_1, _2) ] | xor_   [ _val = _1 ];
        xor_ = (and_ >> "xor" >> and_) [ _val = phx::construct<binop<op_xor>>(_1, _2) ] | and_   [ _val = _1 ];
        and_ = (not_ >> "and" >> not_) [ _val = phx::construct<binop<op_and>>(_1, _2) ] | not_   [ _val = _1 ];
        not_ = ("not" > simple       ) [ _val = phx::construct<unop <op_not>>(_1)     ] | simple [ _val = _1 ];

        simple = (('(' > expr_ > ')') | var_);
        var_ = qi::lexeme[ +alpha ];
    }

  private:
    qi::rule<It, var() , Skipper> var_;
    qi::rule<It, expr(), Skipper> not_, and_, xor_, or_, simple, expr_;
};

Traversing a recursive variant may look cryptic at first, but the boost::static_visitor<> is surprisingly simple once you get the hang of it:

struct printer : boost::static_visitor<void>
{
    printer(std::ostream& os) : _os(os) {}
    std::ostream& _os;

    //
    void operator()(const var& v) const { _os << v; }

    void operator()(const binop<op_and>& b) const { print(" & ", b.oper1, b.oper2); }
    void operator()(const binop<op_or >& b) const { print(" | ", b.oper1, b.oper2); }
    void operator()(const binop<op_xor>& b) const { print(" ^ ", b.oper1, b.oper2); }

    void print(const std::string& op, const expr& l, const expr& r) const
    {
        _os << "(";
            boost::apply_visitor(*this, l);
            _os << op;
            boost::apply_visitor(*this, r);
        _os << ")";
    }

    void operator()(const unop<op_not>& u) const
    {
        _os << "(";
            _os << "!";
            boost::apply_visitor(*this, u.oper1);
        _os << ")";
    }
};

std::ostream& operator<<(std::ostream& os, const expr& e)
{ boost::apply_visitor(printer(os), e); return os; }

For the test cases in the code, the following is output, demonstrating correct handling of the precedence rules by adding (redundant) parentheses:

result: ((a & b) ^ ((c & d) | (a & b)))
result: ((a & b) ^ ((c & d) | (a & b)))
result: (a & b)
result: (a | b)
result: (a ^ b)
result: (!a)
result: ((!a) & b)
result: (!(a & b))
result: (a | (b | c))
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <boost/variant/recursive_wrapper.hpp>

namespace qi    = boost::spirit::qi;
namespace phx   = boost::phoenix;

struct op_or  {};
struct op_and {};
struct op_xor {};
struct op_not {};

typedef std::string var;
template <typename tag> struct binop;
template <typename tag> struct unop;

typedef boost::variant<var, 
        boost::recursive_wrapper<unop <op_not> >, 
        boost::recursive_wrapper<binop<op_and> >,
        boost::recursive_wrapper<binop<op_xor> >,
        boost::recursive_wrapper<binop<op_or> >
        > expr;

template <typename tag> struct binop 
{ 
    explicit binop(const expr& l, const expr& r) : oper1(l), oper2(r) { }
    expr oper1, oper2; 
};

template <typename tag> struct unop  
{ 
    explicit unop(const expr& o) : oper1(o) { }
    expr oper1; 
};

struct printer : boost::static_visitor<void>
{
    printer(std::ostream& os) : _os(os) {}
    std::ostream& _os;

    //
    void operator()(const var& v) const { _os << v; }

    void operator()(const binop<op_and>& b) const { print(" & ", b.oper1, b.oper2); }
    void operator()(const binop<op_or >& b) const { print(" | ", b.oper1, b.oper2); }
    void operator()(const binop<op_xor>& b) const { print(" ^ ", b.oper1, b.oper2); }

    void print(const std::string& op, const expr& l, const expr& r) const
    {
        _os << "(";
            boost::apply_visitor(*this, l);
            _os << op;
            boost::apply_visitor(*this, r);
        _os << ")";
    }

    void operator()(const unop<op_not>& u) const
    {
        _os << "(";
            _os << "!";
            boost::apply_visitor(*this, u.oper1);
        _os << ")";
    }
};

std::ostream& operator<<(std::ostream& os, const expr& e)
{ boost::apply_visitor(printer(os), e); return os; }

template <typename It, typename Skipper = qi::space_type>
    struct parser : qi::grammar<It, expr(), Skipper>
{
    parser() : parser::base_type(expr_)
    {
        using namespace qi;

        expr_  = or_.alias();

        or_  = (xor_ >> "or"  >> or_ ) [ _val = phx::construct<binop<op_or >>(_1, _2) ] | xor_   [ _val = _1 ];
        xor_ = (and_ >> "xor" >> xor_) [ _val = phx::construct<binop<op_xor>>(_1, _2) ] | and_   [ _val = _1 ];
        and_ = (not_ >> "and" >> and_) [ _val = phx::construct<binop<op_and>>(_1, _2) ] | not_   [ _val = _1 ];
        not_ = ("not" > simple       ) [ _val = phx::construct<unop <op_not>>(_1)     ] | simple [ _val = _1 ];

        simple = (('(' > expr_ > ')') | var_);
        var_ = qi::lexeme[ +alpha ];

        BOOST_SPIRIT_DEBUG_NODE(expr_);
        BOOST_SPIRIT_DEBUG_NODE(or_);
        BOOST_SPIRIT_DEBUG_NODE(xor_);
        BOOST_SPIRIT_DEBUG_NODE(and_);
        BOOST_SPIRIT_DEBUG_NODE(not_);
        BOOST_SPIRIT_DEBUG_NODE(simple);
        BOOST_SPIRIT_DEBUG_NODE(var_);
    }

  private:
    qi::rule<It, var() , Skipper> var_;
    qi::rule<It, expr(), Skipper> not_, and_, xor_, or_, simple, expr_;
};

int main()
{
    for (auto& input : std::list<std::string> {
            // From the OP:
            "(a and b) xor ((c and d) or (a and b));",
            "a and b xor (c and d or a and b);",

            /// Simpler tests:
            "a and b;",
            "a or b;",
            "a xor b;",
            "not a;",
            "not a and b;",
            "not (a and b);",
            "a or b or c;",
            })
    {
        auto f(std::begin(input)), l(std::end(input));
        parser<decltype(f)> p;

        try
        {
            expr result;
            bool ok = qi::phrase_parse(f,l,p > ';',qi::space,result);

            if (!ok)
                std::cerr << "invalid input\n";
            else
                std::cout << "result: " << result << "\n";

        } catch (const qi::expectation_failure<decltype(f)>& e)
        {
            std::cerr << "expectation_failure at '" << std::string(e.first, e.last) << "'\n";
        }

        if (f!=l) std::cerr << "unparsed: '" << std::string(f,l) << "'\n";
    }

    return 0;
}

For bonus points, to get a tree exactly like shown in the OP:

static const char indentstep[] = "    ";

struct tree_print : boost::static_visitor<void>
{
    tree_print(std::ostream& os, const std::string& indent=indentstep) : _os(os), _indent(indent) {}
    std::ostream& _os;
    std::string _indent;

    void operator()(const var& v) const { _os << _indent << v << std::endl; }

    void operator()(const binop<op_and>& b) const { print("and ", b.oper1, b.oper2); }
    void operator()(const binop<op_or >& b) const { print("or  ", b.oper2, b.oper1); }
    void operator()(const binop<op_xor>& b) const { print("xor ", b.oper2, b.oper1); }

    void print(const std::string& op, const expr& l, const expr& r) const
    {
        boost::apply_visitor(tree_print(_os, _indent+indentstep), l);
        _os << _indent << op << std::endl;
        boost::apply_visitor(tree_print(_os, _indent+indentstep), r);
    }

    void operator()(const unop<op_not>& u) const
    {
        _os << _indent << "!";
        boost::apply_visitor(tree_print(_os, _indent+indentstep), u.oper1);
    }
};

std::ostream& operator<<(std::ostream& os, const expr& e)
{ 
    boost::apply_visitor(tree_print(os), e); return os; 
}
a
        and 
            b
    or  
            c
        and 
            d
xor 
        a
    and 
        b

as a bonus, added tree_print visitor that mimicks the ascii layout from the OP :)

Your command of Spirit keeps impressing me.

Hi, that is a great example you have here. There is one thing I couldn't figure out. If you try to pass "a or b or c" it fails. Can you post how to fix your code for such cases?

@SergejAndrejev nice observation. I fixed it by providing right recursion for the binary operators (and_, or_, xor_) and added your testcase. Thanks for the heads up

Added an evaluation visitor in another answer, in case people are interested: How to calculate boolean expression in Spirit

parsing - Boolean expression (grammar) parser in c++ - Stack Overflow

c++ parsing boost-spirit
Rectangle 27 80

Operating on the syntax tree

Here is an implementation based on Boost Spirit.

Because Boost Spirit generates recursive descent parsers based on expression templates, honouring the 'idiosyncratic' (sic) precedence rules (as mentioned by others) is quite tedious. Therefore the grammar lacks a certain elegance.

I defined a tree data structure using Boost Variant's recursive variant support, note the definition of expr:

struct op_or  {}; // tag
struct op_and {}; // tag
struct op_xor {}; // tag
struct op_not {}; // tag

typedef std::string var;
template <typename tag> struct binop;
template <typename tag> struct unop;

typedef boost::variant<var, 
        boost::recursive_wrapper<unop <op_not> >, 
        boost::recursive_wrapper<binop<op_and> >,
        boost::recursive_wrapper<binop<op_xor> >,
        boost::recursive_wrapper<binop<op_or> >
        > expr;

The following is the (slightly tedious) grammar definition, as mentioned.

Although I don't consider this grammar optimal, it is quite readable, and we have ourselves a statically compiled parser with strongly typed AST datatype in roughly 50 lines of code. Things could be considerably worse.

template <typename It, typename Skipper = qi::space_type>
    struct parser : qi::grammar<It, expr(), Skipper>
{
    parser() : parser::base_type(expr_)
    {
        using namespace qi;
        expr_  = or_.alias();

        or_  = (xor_ >> "or"  >> xor_) [ _val = phx::construct<binop<op_or >>(_1, _2) ] | xor_   [ _val = _1 ];
        xor_ = (and_ >> "xor" >> and_) [ _val = phx::construct<binop<op_xor>>(_1, _2) ] | and_   [ _val = _1 ];
        and_ = (not_ >> "and" >> not_) [ _val = phx::construct<binop<op_and>>(_1, _2) ] | not_   [ _val = _1 ];
        not_ = ("not" > simple       ) [ _val = phx::construct<unop <op_not>>(_1)     ] | simple [ _val = _1 ];

        simple = (('(' > expr_ > ')') | var_);
        var_ = qi::lexeme[ +alpha ];
    }

  private:
    qi::rule<It, var() , Skipper> var_;
    qi::rule<It, expr(), Skipper> not_, and_, xor_, or_, simple, expr_;
};

Traversing a recursive variant may look cryptic at first, but the boost::static_visitor<> is surprisingly simple once you get the hang of it:

struct printer : boost::static_visitor<void>
{
    printer(std::ostream& os) : _os(os) {}
    std::ostream& _os;

    //
    void operator()(const var& v) const { _os << v; }

    void operator()(const binop<op_and>& b) const { print(" & ", b.oper1, b.oper2); }
    void operator()(const binop<op_or >& b) const { print(" | ", b.oper1, b.oper2); }
    void operator()(const binop<op_xor>& b) const { print(" ^ ", b.oper1, b.oper2); }

    void print(const std::string& op, const expr& l, const expr& r) const
    {
        _os << "(";
            boost::apply_visitor(*this, l);
            _os << op;
            boost::apply_visitor(*this, r);
        _os << ")";
    }

    void operator()(const unop<op_not>& u) const
    {
        _os << "(";
            _os << "!";
            boost::apply_visitor(*this, u.oper1);
        _os << ")";
    }
};

std::ostream& operator<<(std::ostream& os, const expr& e)
{ boost::apply_visitor(printer(os), e); return os; }

For the test cases in the code, the following is output, demonstrating correct handling of the precedence rules by adding (redundant) parentheses:

result: ((a & b) ^ ((c & d) | (a & b)))
result: ((a & b) ^ ((c & d) | (a & b)))
result: (a & b)
result: (a | b)
result: (a ^ b)
result: (!a)
result: ((!a) & b)
result: (!(a & b))
result: (a | (b | c))
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <boost/variant/recursive_wrapper.hpp>

namespace qi    = boost::spirit::qi;
namespace phx   = boost::phoenix;

struct op_or  {};
struct op_and {};
struct op_xor {};
struct op_not {};

typedef std::string var;
template <typename tag> struct binop;
template <typename tag> struct unop;

typedef boost::variant<var, 
        boost::recursive_wrapper<unop <op_not> >, 
        boost::recursive_wrapper<binop<op_and> >,
        boost::recursive_wrapper<binop<op_xor> >,
        boost::recursive_wrapper<binop<op_or> >
        > expr;

template <typename tag> struct binop 
{ 
    explicit binop(const expr& l, const expr& r) : oper1(l), oper2(r) { }
    expr oper1, oper2; 
};

template <typename tag> struct unop  
{ 
    explicit unop(const expr& o) : oper1(o) { }
    expr oper1; 
};

struct printer : boost::static_visitor<void>
{
    printer(std::ostream& os) : _os(os) {}
    std::ostream& _os;

    //
    void operator()(const var& v) const { _os << v; }

    void operator()(const binop<op_and>& b) const { print(" & ", b.oper1, b.oper2); }
    void operator()(const binop<op_or >& b) const { print(" | ", b.oper1, b.oper2); }
    void operator()(const binop<op_xor>& b) const { print(" ^ ", b.oper1, b.oper2); }

    void print(const std::string& op, const expr& l, const expr& r) const
    {
        _os << "(";
            boost::apply_visitor(*this, l);
            _os << op;
            boost::apply_visitor(*this, r);
        _os << ")";
    }

    void operator()(const unop<op_not>& u) const
    {
        _os << "(";
            _os << "!";
            boost::apply_visitor(*this, u.oper1);
        _os << ")";
    }
};

std::ostream& operator<<(std::ostream& os, const expr& e)
{ boost::apply_visitor(printer(os), e); return os; }

template <typename It, typename Skipper = qi::space_type>
    struct parser : qi::grammar<It, expr(), Skipper>
{
    parser() : parser::base_type(expr_)
    {
        using namespace qi;

        expr_  = or_.alias();

        or_  = (xor_ >> "or"  >> or_ ) [ _val = phx::construct<binop<op_or >>(_1, _2) ] | xor_   [ _val = _1 ];
        xor_ = (and_ >> "xor" >> xor_) [ _val = phx::construct<binop<op_xor>>(_1, _2) ] | and_   [ _val = _1 ];
        and_ = (not_ >> "and" >> and_) [ _val = phx::construct<binop<op_and>>(_1, _2) ] | not_   [ _val = _1 ];
        not_ = ("not" > simple       ) [ _val = phx::construct<unop <op_not>>(_1)     ] | simple [ _val = _1 ];

        simple = (('(' > expr_ > ')') | var_);
        var_ = qi::lexeme[ +alpha ];

        BOOST_SPIRIT_DEBUG_NODE(expr_);
        BOOST_SPIRIT_DEBUG_NODE(or_);
        BOOST_SPIRIT_DEBUG_NODE(xor_);
        BOOST_SPIRIT_DEBUG_NODE(and_);
        BOOST_SPIRIT_DEBUG_NODE(not_);
        BOOST_SPIRIT_DEBUG_NODE(simple);
        BOOST_SPIRIT_DEBUG_NODE(var_);
    }

  private:
    qi::rule<It, var() , Skipper> var_;
    qi::rule<It, expr(), Skipper> not_, and_, xor_, or_, simple, expr_;
};

int main()
{
    for (auto& input : std::list<std::string> {
            // From the OP:
            "(a and b) xor ((c and d) or (a and b));",
            "a and b xor (c and d or a and b);",

            /// Simpler tests:
            "a and b;",
            "a or b;",
            "a xor b;",
            "not a;",
            "not a and b;",
            "not (a and b);",
            "a or b or c;",
            })
    {
        auto f(std::begin(input)), l(std::end(input));
        parser<decltype(f)> p;

        try
        {
            expr result;
            bool ok = qi::phrase_parse(f,l,p > ';',qi::space,result);

            if (!ok)
                std::cerr << "invalid input\n";
            else
                std::cout << "result: " << result << "\n";

        } catch (const qi::expectation_failure<decltype(f)>& e)
        {
            std::cerr << "expectation_failure at '" << std::string(e.first, e.last) << "'\n";
        }

        if (f!=l) std::cerr << "unparsed: '" << std::string(f,l) << "'\n";
    }

    return 0;
}

For bonus points, to get a tree exactly like shown in the OP:

static const char indentstep[] = "    ";

struct tree_print : boost::static_visitor<void>
{
    tree_print(std::ostream& os, const std::string& indent=indentstep) : _os(os), _indent(indent) {}
    std::ostream& _os;
    std::string _indent;

    void operator()(const var& v) const { _os << _indent << v << std::endl; }

    void operator()(const binop<op_and>& b) const { print("and ", b.oper1, b.oper2); }
    void operator()(const binop<op_or >& b) const { print("or  ", b.oper2, b.oper1); }
    void operator()(const binop<op_xor>& b) const { print("xor ", b.oper2, b.oper1); }

    void print(const std::string& op, const expr& l, const expr& r) const
    {
        boost::apply_visitor(tree_print(_os, _indent+indentstep), l);
        _os << _indent << op << std::endl;
        boost::apply_visitor(tree_print(_os, _indent+indentstep), r);
    }

    void operator()(const unop<op_not>& u) const
    {
        _os << _indent << "!";
        boost::apply_visitor(tree_print(_os, _indent+indentstep), u.oper1);
    }
};

std::ostream& operator<<(std::ostream& os, const expr& e)
{ 
    boost::apply_visitor(tree_print(os), e); return os; 
}
a
        and 
            b
    or  
            c
        and 
            d
xor 
        a
    and 
        b

as a bonus, added tree_print visitor that mimicks the ascii layout from the OP :)

Your command of Spirit keeps impressing me.

Hi, that is a great example you have here. There is one thing I couldn't figure out. If you try to pass "a or b or c" it fails. Can you post how to fix your code for such cases?

@SergejAndrejev nice observation. I fixed it by providing right recursion for the binary operators (and_, or_, xor_) and added your testcase. Thanks for the heads up

Added an evaluation visitor in another answer, in case people are interested: How to calculate boolean expression in Spirit

parsing - Boolean expression (grammar) parser in c++ - Stack Overflow

c++ parsing boost-spirit
Rectangle 27 80

Operating on the syntax tree

Here is an implementation based on Boost Spirit.

Because Boost Spirit generates recursive descent parsers based on expression templates, honouring the 'idiosyncratic' (sic) precedence rules (as mentioned by others) is quite tedious. Therefore the grammar lacks a certain elegance.

I defined a tree data structure using Boost Variant's recursive variant support, note the definition of expr:

struct op_or  {}; // tag
struct op_and {}; // tag
struct op_xor {}; // tag
struct op_not {}; // tag

typedef std::string var;
template <typename tag> struct binop;
template <typename tag> struct unop;

typedef boost::variant<var, 
        boost::recursive_wrapper<unop <op_not> >, 
        boost::recursive_wrapper<binop<op_and> >,
        boost::recursive_wrapper<binop<op_xor> >,
        boost::recursive_wrapper<binop<op_or> >
        > expr;

The following is the (slightly tedious) grammar definition, as mentioned.

Although I don't consider this grammar optimal, it is quite readable, and we have ourselves a statically compiled parser with strongly typed AST datatype in roughly 50 lines of code. Things could be considerably worse.

template <typename It, typename Skipper = qi::space_type>
    struct parser : qi::grammar<It, expr(), Skipper>
{
    parser() : parser::base_type(expr_)
    {
        using namespace qi;
        expr_  = or_.alias();

        or_  = (xor_ >> "or"  >> xor_) [ _val = phx::construct<binop<op_or >>(_1, _2) ] | xor_   [ _val = _1 ];
        xor_ = (and_ >> "xor" >> and_) [ _val = phx::construct<binop<op_xor>>(_1, _2) ] | and_   [ _val = _1 ];
        and_ = (not_ >> "and" >> not_) [ _val = phx::construct<binop<op_and>>(_1, _2) ] | not_   [ _val = _1 ];
        not_ = ("not" > simple       ) [ _val = phx::construct<unop <op_not>>(_1)     ] | simple [ _val = _1 ];

        simple = (('(' > expr_ > ')') | var_);
        var_ = qi::lexeme[ +alpha ];
    }

  private:
    qi::rule<It, var() , Skipper> var_;
    qi::rule<It, expr(), Skipper> not_, and_, xor_, or_, simple, expr_;
};

Traversing a recursive variant may look cryptic at first, but the boost::static_visitor<> is surprisingly simple once you get the hang of it:

struct printer : boost::static_visitor<void>
{
    printer(std::ostream& os) : _os(os) {}
    std::ostream& _os;

    //
    void operator()(const var& v) const { _os << v; }

    void operator()(const binop<op_and>& b) const { print(" & ", b.oper1, b.oper2); }
    void operator()(const binop<op_or >& b) const { print(" | ", b.oper1, b.oper2); }
    void operator()(const binop<op_xor>& b) const { print(" ^ ", b.oper1, b.oper2); }

    void print(const std::string& op, const expr& l, const expr& r) const
    {
        _os << "(";
            boost::apply_visitor(*this, l);
            _os << op;
            boost::apply_visitor(*this, r);
        _os << ")";
    }

    void operator()(const unop<op_not>& u) const
    {
        _os << "(";
            _os << "!";
            boost::apply_visitor(*this, u.oper1);
        _os << ")";
    }
};

std::ostream& operator<<(std::ostream& os, const expr& e)
{ boost::apply_visitor(printer(os), e); return os; }

For the test cases in the code, the following is output, demonstrating correct handling of the precedence rules by adding (redundant) parentheses:

result: ((a & b) ^ ((c & d) | (a & b)))
result: ((a & b) ^ ((c & d) | (a & b)))
result: (a & b)
result: (a | b)
result: (a ^ b)
result: (!a)
result: ((!a) & b)
result: (!(a & b))
result: (a | (b | c))
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <boost/variant/recursive_wrapper.hpp>

namespace qi    = boost::spirit::qi;
namespace phx   = boost::phoenix;

struct op_or  {};
struct op_and {};
struct op_xor {};
struct op_not {};

typedef std::string var;
template <typename tag> struct binop;
template <typename tag> struct unop;

typedef boost::variant<var, 
        boost::recursive_wrapper<unop <op_not> >, 
        boost::recursive_wrapper<binop<op_and> >,
        boost::recursive_wrapper<binop<op_xor> >,
        boost::recursive_wrapper<binop<op_or> >
        > expr;

template <typename tag> struct binop 
{ 
    explicit binop(const expr& l, const expr& r) : oper1(l), oper2(r) { }
    expr oper1, oper2; 
};

template <typename tag> struct unop  
{ 
    explicit unop(const expr& o) : oper1(o) { }
    expr oper1; 
};

struct printer : boost::static_visitor<void>
{
    printer(std::ostream& os) : _os(os) {}
    std::ostream& _os;

    //
    void operator()(const var& v) const { _os << v; }

    void operator()(const binop<op_and>& b) const { print(" & ", b.oper1, b.oper2); }
    void operator()(const binop<op_or >& b) const { print(" | ", b.oper1, b.oper2); }
    void operator()(const binop<op_xor>& b) const { print(" ^ ", b.oper1, b.oper2); }

    void print(const std::string& op, const expr& l, const expr& r) const
    {
        _os << "(";
            boost::apply_visitor(*this, l);
            _os << op;
            boost::apply_visitor(*this, r);
        _os << ")";
    }

    void operator()(const unop<op_not>& u) const
    {
        _os << "(";
            _os << "!";
            boost::apply_visitor(*this, u.oper1);
        _os << ")";
    }
};

std::ostream& operator<<(std::ostream& os, const expr& e)
{ boost::apply_visitor(printer(os), e); return os; }

template <typename It, typename Skipper = qi::space_type>
    struct parser : qi::grammar<It, expr(), Skipper>
{
    parser() : parser::base_type(expr_)
    {
        using namespace qi;

        expr_  = or_.alias();

        or_  = (xor_ >> "or"  >> or_ ) [ _val = phx::construct<binop<op_or >>(_1, _2) ] | xor_   [ _val = _1 ];
        xor_ = (and_ >> "xor" >> xor_) [ _val = phx::construct<binop<op_xor>>(_1, _2) ] | and_   [ _val = _1 ];
        and_ = (not_ >> "and" >> and_) [ _val = phx::construct<binop<op_and>>(_1, _2) ] | not_   [ _val = _1 ];
        not_ = ("not" > simple       ) [ _val = phx::construct<unop <op_not>>(_1)     ] | simple [ _val = _1 ];

        simple = (('(' > expr_ > ')') | var_);
        var_ = qi::lexeme[ +alpha ];

        BOOST_SPIRIT_DEBUG_NODE(expr_);
        BOOST_SPIRIT_DEBUG_NODE(or_);
        BOOST_SPIRIT_DEBUG_NODE(xor_);
        BOOST_SPIRIT_DEBUG_NODE(and_);
        BOOST_SPIRIT_DEBUG_NODE(not_);
        BOOST_SPIRIT_DEBUG_NODE(simple);
        BOOST_SPIRIT_DEBUG_NODE(var_);
    }

  private:
    qi::rule<It, var() , Skipper> var_;
    qi::rule<It, expr(), Skipper> not_, and_, xor_, or_, simple, expr_;
};

int main()
{
    for (auto& input : std::list<std::string> {
            // From the OP:
            "(a and b) xor ((c and d) or (a and b));",
            "a and b xor (c and d or a and b);",

            /// Simpler tests:
            "a and b;",
            "a or b;",
            "a xor b;",
            "not a;",
            "not a and b;",
            "not (a and b);",
            "a or b or c;",
            })
    {
        auto f(std::begin(input)), l(std::end(input));
        parser<decltype(f)> p;

        try
        {
            expr result;
            bool ok = qi::phrase_parse(f,l,p > ';',qi::space,result);

            if (!ok)
                std::cerr << "invalid input\n";
            else
                std::cout << "result: " << result << "\n";

        } catch (const qi::expectation_failure<decltype(f)>& e)
        {
            std::cerr << "expectation_failure at '" << std::string(e.first, e.last) << "'\n";
        }

        if (f!=l) std::cerr << "unparsed: '" << std::string(f,l) << "'\n";
    }

    return 0;
}

For bonus points, to get a tree exactly like shown in the OP:

static const char indentstep[] = "    ";

struct tree_print : boost::static_visitor<void>
{
    tree_print(std::ostream& os, const std::string& indent=indentstep) : _os(os), _indent(indent) {}
    std::ostream& _os;
    std::string _indent;

    void operator()(const var& v) const { _os << _indent << v << std::endl; }

    void operator()(const binop<op_and>& b) const { print("and ", b.oper1, b.oper2); }
    void operator()(const binop<op_or >& b) const { print("or  ", b.oper2, b.oper1); }
    void operator()(const binop<op_xor>& b) const { print("xor ", b.oper2, b.oper1); }

    void print(const std::string& op, const expr& l, const expr& r) const
    {
        boost::apply_visitor(tree_print(_os, _indent+indentstep), l);
        _os << _indent << op << std::endl;
        boost::apply_visitor(tree_print(_os, _indent+indentstep), r);
    }

    void operator()(const unop<op_not>& u) const
    {
        _os << _indent << "!";
        boost::apply_visitor(tree_print(_os, _indent+indentstep), u.oper1);
    }
};

std::ostream& operator<<(std::ostream& os, const expr& e)
{ 
    boost::apply_visitor(tree_print(os), e); return os; 
}
a
        and 
            b
    or  
            c
        and 
            d
xor 
        a
    and 
        b

as a bonus, added tree_print visitor that mimicks the ascii layout from the OP :)

Your command of Spirit keeps impressing me.

Hi, that is a great example you have here. There is one thing I couldn't figure out. If you try to pass "a or b or c" it fails. Can you post how to fix your code for such cases?

@SergejAndrejev nice observation. I fixed it by providing right recursion for the binary operators (and_, or_, xor_) and added your testcase. Thanks for the heads up

Added an evaluation visitor in another answer, in case people are interested: How to calculate boolean expression in Spirit

parsing - Boolean expression (grammar) parser in c++ - Stack Overflow

c++ parsing boost-spirit
Rectangle 27 21

Microsoft has given us a cleaner, more convenient way of creating anonymous delegates called Lambda expressions. However, there is not a lot of attention being paid to the expressions portion of this statement. Microsoft released a entire namespace, System.Linq.Expressions, which contains classes to create expression trees based on lambda expressions. Expression trees are made up of objects that represent logic. For example, x = y + z is an expression that might be part of an expression tree in .Net. Consider the following (simple) example:

using System;
using System.Linq;
using System.Linq.Expressions;


namespace ExpressionTreeThingy
{
    class Program
    {
        static void Main(string[] args)
        {
            Expression<Func<int, int>> expr = (x) => x + 1; //this is not a delegate, but an object
            var del = expr.Compile(); //compiles the object to a CLR delegate, at runtime
            Console.WriteLine(del(5)); //we are just invoking a delegate at this point
            Console.ReadKey();
        }
    }
}

This example is trivial. And I am sure you are thinking, "This is useless as I could have directly created the delegate instead of creating an expression and compiling it at runtime". And you would be right. But this provides the foundation for expression trees. There are a number of expressions available in the Expressions namespaces, and you can build your own. I think you can see that this might be useful when you don't know exactly what the algorithm should be at design or compile time. I saw an example somewhere for using this to write a scientific calculator. You could also use it for Bayesian systems, or for genetic programming (AI). A few times in my career I have had to write Excel-like functionality that allowed users to enter simple expressions (addition, subtrations, etc) to operate on available data. In pre-.Net 3.5 I have had to resort to some scripting language external to C#, or had to use the code-emitting functionality in reflection to create .Net code on the fly. Now I would use expression trees.

c# 3.0 - C# Lambda expressions: Why should I use them? - Stack Overflo...

c# c#-3.0 lambda
Rectangle 27 21

Basically as far as C# is concerned, lambda expressions are an easy way to create a delegate (or an expression tree, but let's leave those aside for now).

In C# 1 we could only create delegate instances from normal methods. In C# 2 we gained anonymous methods. In C# 3 we gained lambda expressions, which are like more concise anonymous methods.

They're particularly concise when you want to express some logic which takes one value and returns a value. For instance, in the context of LINQ:

// Only include children - a predicate
var query = dataSource.Where(person => person.Age < 18) 
                       // Transform to sequence of names - a projection
                      .Select(person => person.Name);

There's a fuller discussion of this - along with other aspects - in my article on closures.

I saw this question and I thought, "Ooo! Maybe I can beat Jon Skeet to the punch!" But then I didn't. :) +1

that code is looking pretty nice, even though I am pondering if Select is the right name. if you used Map, you'd arrive at relational algebra in code. :)

To me, Map suggests the creation of a map from key to value, where you can look up by key. Personally I'd prefer Project().

There was a brief, tangentially related discussion on Lippert's blog earlier this year about reading the "=>" operator: blogs.msdn.com/ericlippert/archive/2008/05/16/

delegates - Can you explain lambda expressions? - Stack Overflow

delegates lambda closures imperative-languages
Rectangle 27 4

Fundamentally, your AST will be constructed like a binary tree where each AST node is a "C" structure that holds both a "left" and "right" pointer. The other elements of the AST are typically context sensitive. For example, a variable declaration versus a function definition or an expression in a function. For the example you cited, the rough tree would mirror this:

+
 /   \
1     *
      /\
     2  3

So by substituting the above nodes 1 + (2 * 3) with your AST construct would be similar to:

-----------------
                | type: ADDOP   |
                | left: struct* |
                | right: struct*|
                -----------------
              /                   \
             /                     \
 (ADDOP left points to)         (ADDOP right points to)
------------------------       --------------------------  
| type: NUMLITERAL     |       | type: MULTOP           |
| value: 1             |       | left: struct*          |
| left: struct* (null) |       | right: struct*         |
| right: struct*(null) |       --------------------------
------------------------              /               \
                                     /                 \

                    (MULTOP left points to)         (MULTOP right points to)
                    ------------------------       --------------------------  
                    | type: NUMLITERAL     |       | type: NUMLITERAL       |
                    | value: 2             |       | value: 3               |
                    | left: struct* (null) |       | left: struct* (null)   |
                    | right: struct*(null) |       | right: struct* (null)  |
                    ------------------------       --------------------------

I assume that you know enough about "C" and how to malloc nodes and assign the left/right pointers.

Now the remaining activity would be to do a post order traversal of the tree to either evaluate the expression and produce a result or to emit the appropriate intermediate code/machine code that aligns to a compiled result. Either choice bringing with it a massive amount of thinking and planning on your part.

BTW: As noted, the AST nodes are going to typically have attributes based on the level of abstraction you want to represent. Also note that a typical compiler may leverage multiple AST for different reasons. Yep, more reading/studying on your part.

Note: This illustrates the data structure for an AST but @mikeb answer is rock solid for how to get the string "1 + 2 * 3" into the nodes of such a structure.

c - Building an interpreter: designing an AST - Stack Overflow

c compiler-construction grammar abstract-syntax-tree interpreter
Rectangle 27 23

Capturing a local variable is actually performed by "hoisting" the local variable into an instance variable of a compiler-generated class. The C# compiler creates a new instance of the extra class at the appropriate time, and changes any access to the local variable into an access of the instance variable in the relevant instance.

So the expression tree then needs to be a field access within the instance - and the instance itself is provided via a ConstantExpression.

The simplest approach for working how to create expression trees is usually to create something similar in a lambda expression, then look at the generated code in Reflector, turning the optimization level down so that Reflector doesn't convert it back to lambda expressions.

var hoistedConstant = Expression.Property(Expression.Constant(new {Value = filterString}), "Value");

@Appetere What about Expression.Constant(filterString)? Admittedly, it wouldn't reflect changes to the variable, but neither would your suggestion.

c# - Local variable and expression trees - Stack Overflow

c# linq lambda
Rectangle 27 4

Use reflection to get from the string property names to a PropertyInfo's. You can then build an expression tree using the PropertyInfo's to dynamically construct all the orderbys. Once you have the expression tree, compile it to a delegate, (say Func, IEnumerable>) Pass in your _list parameter to this delegate and it will give you the ordered result as another enumerable.

To get the reflection information for the generic method on Enumerable, have a look at the answer on this post: Get a generic method without using GetMethods

public static class Helper
{
    public static IEnumerable<T> BuildOrderBys<T>(
        this IEnumerable<T> source,
        params SortDescription[] properties)
    {
        if (properties == null || properties.Length == 0) return source;

        var typeOfT = typeof (T);

        Type t = typeOfT;

        IOrderedEnumerable<T> result = null;
        var thenBy = false;

        foreach (var item in properties
            .Select(prop => new {PropertyInfo = t.GetProperty(prop.PropertyName), prop.Direction}))
        {
            var oExpr = Expression.Parameter(typeOfT, "o");
            var propertyInfo = item.PropertyInfo;
            var propertyType = propertyInfo.PropertyType;
            var isAscending = item.Direction == ListSortDirection.Ascending;

            if (thenBy)
            {
                var prevExpr = Expression.Parameter(typeof (IOrderedEnumerable<T>), "prevExpr");
                var expr1 = Expression.Lambda<Func<IOrderedEnumerable<T>, IOrderedEnumerable<T>>>(
                    Expression.Call(
                        (isAscending ? thenByMethod : thenByDescendingMethod).MakeGenericMethod(typeOfT, propertyType),
                        prevExpr,
                        Expression.Lambda(
                            typeof (Func<,>).MakeGenericType(typeOfT, propertyType),
                            Expression.MakeMemberAccess(oExpr, propertyInfo),
                            oExpr)
                        ),
                    prevExpr)
                    .Compile();

                result = expr1(result);
            }
            else
            {
                var prevExpr = Expression.Parameter(typeof (IEnumerable<T>), "prevExpr");
                var expr1 = Expression.Lambda<Func<IEnumerable<T>, IOrderedEnumerable<T>>>(
                    Expression.Call(
                        (isAscending ? orderByMethod : orderByDescendingMethod).MakeGenericMethod(typeOfT, propertyType),
                        prevExpr,
                        Expression.Lambda(
                            typeof (Func<,>).MakeGenericType(typeOfT, propertyType),
                            Expression.MakeMemberAccess(oExpr, propertyInfo),
                            oExpr)
                        ),
                    prevExpr)
                    .Compile();

                result = expr1(source);
                thenBy = true;
            }
        }
        return result;
    }

    private static MethodInfo orderByMethod =
        MethodOf(() => Enumerable.OrderBy(default(IEnumerable<object>), default(Func<object, object>)))
            .GetGenericMethodDefinition();

    private static MethodInfo orderByDescendingMethod =
        MethodOf(() => Enumerable.OrderByDescending(default(IEnumerable<object>), default(Func<object, object>)))
            .GetGenericMethodDefinition();

    private static MethodInfo thenByMethod =
        MethodOf(() => Enumerable.ThenBy(default(IOrderedEnumerable<object>), default(Func<object, object>)))
            .GetGenericMethodDefinition();

    private static MethodInfo thenByDescendingMethod =
        MethodOf(() => Enumerable.ThenByDescending(default(IOrderedEnumerable<object>), default(Func<object, object>)))
            .GetGenericMethodDefinition();

    public static MethodInfo MethodOf<T>(Expression<Func<T>> method)
    {
        MethodCallExpression mce = (MethodCallExpression) method.Body;
        MethodInfo mi = mce.Method;
        return mi;
    }
}

public static class Sample
{
    private static void Main()
    {
      var data = new List<Customer>
        {
          new Customer {ID = 3, Name = "a"},
          new Customer {ID = 3, Name = "c"},
          new Customer {ID = 4},
          new Customer {ID = 3, Name = "b"},
          new Customer {ID = 2}
        };

      var result = data.BuildOrderBys(
        new SortDescription("ID", ListSortDirection.Ascending),
        new SortDescription("Name", ListSortDirection.Ascending)
        ).Dump();
    }
}

public class Customer
{
    public int ID { get; set; }
    public string Name { get; set; }
}

The result of the sample as shown in LinqPad

In my sample I recursively build an expression and compile per property, but you can construct the entire expression tree with Expression.Block and then only compile once.

You might also want to expand on this sample to include a bool[] ascending parameter if you want to make some properties sort descending.

Your solution looks very good. Thank you. Just one thing, do you have any ideas on integrating support for ASC and DESC sorting? I added two more MethodInfo, but I can't find how to move forward from that.

I updated the sample as per your request. The properties parameter is now a System.ComponentModel.SortDescription[]. It contains both the property name and an order direction.

c# - Dynamic chaining of List orderby - Stack Overflow

c# linq generics
Rectangle 27 4

Although it's not very 'C-sharpey' to switch on type, I know that construct would be pretty helpful in general use - I have at least one personal project that could use it (although its managable ATM). Is there much of a compile performance problem, with the expression tree re-writing?

Not if you cache the object for re-use (which is largely how C# lambda expressions work, except the compiler hides the code). The re-writing definitely improves the compiled performance - however, for regular use (rather than LINQ-to-Something) I expect the delegate version might be more useful.

Note also - it isn't necessarily a switch on type - it could also be used as a composite conditional (even thru LINQ) - but without a messy x=> Test ? Result1 : (Test2 ? Result2 : (Test3 ? Result 3 : Result4))

Nice to know, although I was meaning the performance of the actual compilation: how long csc.exe takes - I'm not familiar enough with C# to know if that is ever really a problem, but it's a big issue for C++.

csc won't blink at this - it is so similar to how LINQ works, and the C# 3.0 compiler is quite good at LINQ/extension methods etc.

c# - switch / pattern matching idea - Stack Overflow

c# switch-statement
Rectangle 27 2

The first query is using Linq-To-Entities. Linq-To-Entities methods build an expression tree that is translated to sql when you enumerate. If you include something it can't translate to sql you will get an exception. ToString() is an example. LINQ to Entities does not recognize the method 'System.String ToString()'

Calling ToList() makes the enumeration occur. So the sql is run, it doesn't include anything that can't be translated, and the data is moved into memory. Now you are using Linq-To-Objects, and standard C# method calls are recognised

var venue = context.VenueEntitySet.FirstOrDefault(r => r.Venue == venueName))

EDIT OK we know that wasn't the problem now - but another idea to consider is getting the data in one call to the database. Something like this:

var query = from band in context.BandsEntitySet
                    //not sure the join makes sense. How come every band has a VenueName?
                    //join venue in context.VenueEntitySet 
                    //on band.VenueName equals venue.Name
                    //surely there should be a navigation property
                    from venue in band.Venues //using a navigation property
                    where band.ID == 12345
                    select new { 
                                   BandName = band.Name, 
                                   VenueName = venue.Name, 
                                   PlayDate = venue.PlayDate, 
                                   Address = venue.Address 
                               };

        foreach (var item in query)
        {                
            Debug.WriteLine(item.BandName + " is playing in " 
                           + item.VenueName + " on the " + item.PlayDate);
            Debug.WriteLine("The address of " + item.VenueName + " is " + item.Address);

        }

And that should also avoid the problem with multiple open DataReaders

Thanks, but that throws the same exception. Any other ideas? I see what you are saying though.

DimAccounts
var venue = bandList.FirstOrDefault (l => l.AccountKey.Equals(15));

@Serv yes my bad. I saw that ToList() solved the problem and thought it was the common problem with LINQ to Entities does not recognize the method There is something else going on..

+1 for doing in in a single operation with a join - that's using yer noggin.

This is interesting, and would greatly improve the readability and efficiency of my code, but by issue is that there can be multiple venues per band. When I run your code, it prints out every VenueName, PlayDate and Address even if it isn't found in the Bands Table. Does that make sense?

c# - Searching For A Record After A LINQ Query Exception - Stack Overf...

c# linq entity-framework
Rectangle 27 4

Just to make sure that I understand this correctly: Your actual problem is to determine the relative influence of each variable within a boolean expression on the outcome of said expression?

That is what I am calculating but my problem is not with how I calculate it logically but with my use of the python eval built-in to perform evaluating.

So, this seems to be a classic XY problem. You have an actual problem which is to determine the relative influence of each variable within the a boolean expression. You have attempted to solve this in a rather ineffective way, and now that you actually feel the inefficiency (in both memory usage and run time), you look for ways to improve your solution instead of looking for better ways to solve your original problem.

In any way, lets first look at how you are trying to solve this. Im not exactly sure what gen_rand_bits is supposed to do, so I cant really take that into account. But still, you are essentially trying out every possible combination of variable assignments and see if flipping the value for a single variable changes the outcome of the formula result. Luckily, these are just boolean variables, so you are only looking at 2^N possible combinations. This means you have exponential run time. Now, O(2^N) algorithms are in theory very very bad, while in practice its often somewhat okay to use them (because most have an acceptable average case and execute fast enough). However, being an exhaustive algorithm, you actually have to look at every single combination and cant shortcut. Plus the compilation and value evaluation using Pythons eval is apparently not so fast to make the inefficient algorithm acceptable.

So, we should look for a different solution. When looking at your solution, one might say that more efficient is not really possible, but when looking at the original problem, we can argue otherwise.

You essentially want to do things similar to what compilers do as static analysis. You want to look at the source code and analyze it just from there without having to actually evaluate that. As the language you are analyzing is highly restricted (being only a boolean expression with very few operators), this isnt really that hard.

Code analysis usually works on the abstract syntax tree (or an augmented version of that). Python offers code analysis and abstract syntax tree generation with its ast module. We can use this to parse the expression and get the AST. Then based on the tree, we can analyze how relevant each part of an expression is for the whole.

Now, evaluating the relevance of each variable can get quite complicated, but you can do it all by analyzing the syntax tree. I will show you a simple evaluation that supports all boolean operators but will not further check the semantic influence of expressions:

import ast

class ExpressionEvaluator:
    def __init__ (self, rawExpression):
        self.raw = rawExpression
        self.ast = ast.parse(rawExpression)

    def run (self):
        return self.evaluate(self.ast.body[0])

    def evaluate (self, expr):
        if isinstance(expr, ast.Expr):
            return self.evaluate(expr.value)
        elif isinstance(expr, ast.Name):
            return self.evaluateName(expr)
        elif isinstance(expr, ast.UnaryOp):
            if isinstance(expr.op, ast.Invert):
                return self.evaluateInvert(expr)
            else:
                raise Exception('Unknown unary operation {}'.format(expr.op))
        elif isinstance(expr, ast.BinOp):
            if isinstance(expr.op, ast.BitOr):
                return self.evaluateBitOr(expr.left, expr.right)
            elif isinstance(expr.op, ast.BitAnd):
                return self.evaluateBitAnd(expr.left, expr.right)
            elif isinstance(expr.op, ast.BitXor):
                return self.evaluateBitXor(expr.left, expr.right)
            else:
                raise Exception('Unknown binary operation {}'.format(expr.op))
        else:
            raise Exception('Unknown expression {}'.format(expr))

    def evaluateName (self, expr):
        return { expr.id: 1 }

    def evaluateInvert (self, expr):
        return self.evaluate(expr.operand)

    def evaluateBitOr (self, left, right):
        return self.join(self.evaluate(left), .5, self.evaluate(right), .5)

    def evaluateBitAnd (self, left, right):
        return self.join(self.evaluate(left), .5, self.evaluate(right), .5)

    def evaluateBitXor (self, left, right):
        return self.join(self.evaluate(left), .5, self.evaluate(right), .5)

    def join (self, a, ratioA, b, ratioB):
        d = { k: v * ratioA for k, v in a.items() }
        for k, v in b.items():
            if k in d:
                d[k] += v * ratioB
            else:
                d[k] = v * ratioB
        return d

expr = '((A&B)|(C&D)^~E)'
ee = ExpressionEvaluator(expr)
print(ee.run())
# > {'A': 0.25, 'C': 0.125, 'B': 0.25, 'E': 0.25, 'D': 0.125}

This implementation will essentially generate a plain AST for the given expression and the recursively walk through the tree and evaluate the different operators. The big evaluate method just delegates the work to the type specific methods below; its similar to what ast.NodeVisitor does except that we return the analyzation results from each node here. One could augment the nodes instead of returning it instead though.

In this case, the evaluation is just based on ocurrence in the expression. I dont explicitely check for semantic effects. So for an expression A | (A & B), I get {'A': 0.75, 'B': 0.25}, although one could argue that semantically B has no relevance at all to the result (making it {'A': 1} instead). This is however something Ill leave for you. As of now, every binary operation is handled identically (each operand getting a relevance of 50%), but that can be of course adjusted to introduce some semantic rules.

I had thought of other ways of solving this problem but it always a lead to a dead end (non-functional code); hence the reason I stuck with the expression analysis. Your response is very insightful though, I will definitely try to modify and implement it to give 'correct' values - EX: expressions such as A | (A & B) and A & (A | B) etc. like you mention. From those expressions it isn't hard to see {'A': 1}, but an expression such as (A & B) | (A & D) would be a little more difficult to arrive at {'A': 0.75, 'B': 0.25, 'D': 0.25}. I'm not sure how I could program that in though.

One solution I had thought of previously was actually similar to your answer. I would start at a base expression and build up; to find the new value of A, for example, I would take A from the previous expression and multiply it by the probability the other input side is non-affecting. Take (A|B)&(C|D) I would multiply A * prob( (C|D) == 1) since (C|D) has to be 1 for A to be able to change the output. To handle A appearing on the other side (A|B) & (A|C) I came up with left(A) * prob( (A|D) == 1) + right(A) * prob( (A|B) == 1). This does not work for all cases, ie (A|B|C) & (A|C)

Faster method of evaluating a boolean expression as a string in Python...

python string boolean expression eval
Rectangle 27 14

The snippets provided provide a way to wrap each LINQ query with your own QueryProvider and IQueryable root. This would mean that you've got to have control over the initial query starting (as you'll have most of the time using any sort of pattern).

The problem with this method is that it's not transparent, a more ideal situation would be to inject something in the entities container at the constructor level.

I've created a compilable the implementation, got it to work with entity framework, and added support for the ObjectQuery.Include method. The expression visitor class can be copied from MSDN.

public class QueryTranslator<T> : IOrderedQueryable<T>
{
    private Expression expression = null;
    private QueryTranslatorProvider<T> provider = null;

    public QueryTranslator(IQueryable source)
    {
        expression = Expression.Constant(this);
        provider = new QueryTranslatorProvider<T>(source);
    }

    public QueryTranslator(IQueryable source, Expression e)
    {
        if (e == null) throw new ArgumentNullException("e");
        expression = e;
        provider = new QueryTranslatorProvider<T>(source);
    }

    public IEnumerator<T> GetEnumerator()
    {
        return ((IEnumerable<T>)provider.ExecuteEnumerable(this.expression)).GetEnumerator();
    }

    System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
    {
        return provider.ExecuteEnumerable(this.expression).GetEnumerator();
    }

    public QueryTranslator<T> Include(String path)
    {
        ObjectQuery<T> possibleObjectQuery = provider.source as ObjectQuery<T>;
        if (possibleObjectQuery != null)
        {
            return new QueryTranslator<T>(possibleObjectQuery.Include(path));
        }
        else
        {
            throw new InvalidOperationException("The Include should only happen at the beginning of a LINQ expression");
        }
    }

    public Type ElementType
    {
        get { return typeof(T); }
    }

    public Expression Expression
    {
        get { return expression; }
    }

    public IQueryProvider Provider
    {
        get { return provider; }
    }
}

public class QueryTranslatorProvider<T> : ExpressionVisitor, IQueryProvider
{
    internal IQueryable source;

    public QueryTranslatorProvider(IQueryable source)
    {
        if (source == null) throw new ArgumentNullException("source");
        this.source = source;
    }

    public IQueryable<TElement> CreateQuery<TElement>(Expression expression)
    {
        if (expression == null) throw new ArgumentNullException("expression");

        return new QueryTranslator<TElement>(source, expression) as IQueryable<TElement>;
    }

    public IQueryable CreateQuery(Expression expression)
    {
        if (expression == null) throw new ArgumentNullException("expression");
        Type elementType = expression.Type.GetGenericArguments().First();
        IQueryable result = (IQueryable)Activator.CreateInstance(typeof(QueryTranslator<>).MakeGenericType(elementType),
            new object[] { source, expression });
        return result;
    }

    public TResult Execute<TResult>(Expression expression)
    {
        if (expression == null) throw new ArgumentNullException("expression");
        object result = (this as IQueryProvider).Execute(expression);
        return (TResult)result;
    }

    public object Execute(Expression expression)
    {
        if (expression == null) throw new ArgumentNullException("expression");

        Expression translated = this.Visit(expression);
        return source.Provider.Execute(translated);
    }

    internal IEnumerable ExecuteEnumerable(Expression expression)
    {
        if (expression == null) throw new ArgumentNullException("expression");

        Expression translated = this.Visit(expression);
        return source.Provider.CreateQuery(translated);
    }

    #region Visitors
    protected override Expression VisitConstant(ConstantExpression c)
    {
        // fix up the Expression tree to work with EF again
        if (c.Type == typeof(QueryTranslator<T>))
        {
            return source.Expression;
        }
        else
        {
            return base.VisitConstant(c);
        }
    }
    #endregion
}

Example usage in your repository:

public IQueryable<User> List()
{
    return new QueryTranslator<User>(entities.Users).Include("Department");
}

No I got it to work, but strange that you left out the part to fix it back to a EF query.

c# - How to wrap Entity Framework to intercept the LINQ expression jus...

c# linq entity-framework expression-trees
Rectangle 27 6

You have to generate an expression tree, but a simple one, so it's not so hard...

void Main()
{
    var source = new[]
    {
        new SourceModelType { A = "hello", B = "world", C = "foo", D = "bar", E = "Baz" },
        new SourceModelType { A = "The", B = "answer", C = "is", D = "42", E = "!" }
    };

    var dest = ProjectionMap<SourceModelType, DestModelType>(source.AsQueryable());
    dest.Dump();
}

public static IQueryable<TDest> ProjectionMap<TSource, TDest>(IQueryable<TSource> sourceModel)
    where TDest : new()
{
    var sourceProperties = typeof(TSource).GetProperties().Where(p => p.CanRead);
    var destProperties =   typeof(TDest).GetProperties().Where(p => p.CanWrite);
    var propertyMap = from d in destProperties
                      join s in sourceProperties on new { d.Name, d.PropertyType } equals new { s.Name, s.PropertyType }
                      select new { Source = s, Dest = d };
    var itemParam = Expression.Parameter(typeof(TSource), "item");
    var memberBindings = propertyMap.Select(p => (MemberBinding)Expression.Bind(p.Dest, Expression.Property(itemParam, p.Source)));
    var newExpression = Expression.New(typeof(TDest));
    var memberInitExpression = Expression.MemberInit(newExpression, memberBindings);
    var projection = Expression.Lambda<Func<TSource, TDest>>(memberInitExpression, itemParam);
    projection.Dump();
    return sourceModel.Select(projection);
}

(tested in LinqPad, hence the Dumps)

The generated projection expression looks like that :

item => new DestModelType() {A = item.A, C = item.C, E = item.E}

Thank you for this solution. I am deep into understanding how it works. If I wanted to have it drill into complex objects, I would have to alter the propertyMap, correct?

If you want to understand how the expression is constructed, I suggest you use LinqPad; it allows you to easily inspect each node of an expression. As for your question, I'm not sure I understand what you mean... if you only know the source and destination types, you can't really do anything more complex than copying properties with the same name.

What if you wanted to incorporate complex objects so that item => new DestModelType() {A = item.A.X, C = item.C, E = item.E}. This could be through a property attribute designating what to map to.

I think that it would be much harder, but still doable... you would have to change propertyMap and memberBindings according to your own logic

mapping - Using LINQ to map dynamically (or construct projections) - S...

linq mapping map-projections