reduce lexer recursion

This does not seem to improve performance, but returning INTERNAL_READ_NEXT_TOKEN from the lexer to indicate that it has a match, but needs to now read the next token will reduce the recursion.