`refex.python.matchers`¶

`base_matchers`¶

Control Flow Matchers¶

class Anything(**kwargs)¶

Matches anything, similar to the regex ..

Also available as _ in --mode=py.

class Unless(**kwargs)¶: Inverts a matcher and discard its bindings.

class AllOf(**kwargs)¶: Matches if and only if all submatchers do, and merges the results.

class AnyOf(**kwargs)¶: Matches if at least one submatcher does, and returns the first result.

class Once(**kwargs)¶

Runs the submatcher at most once successfully.

Matches if the submatcher has ever matched, including in this run. Fails if the matcher has not ever matched.

If key is provided, then any other Once() with the same key shares state, and is considered equivalent for the sake of the above.

Examples¶

These matchers will match anything:

Anything()
Unless(Unless(Anything()))
AllOf(Anything(), Anything())
AllOf(Anything())
AllOf()

These matchers will match nothing:

Unless(Anything())
AllOf(Unless(Anything()))
AnyOf()

Binding Manipulation Matchers¶

class Bind(**kwargs)¶

Binds an AST-matcher expression to a name in the result.

Parameters

name – The name to bind to. Valid names must be words that don’t begin with a double-underscore (__).
submatcher – The matcher whose result will be bound to name.
on_conflict – A conflict resolution strategy. Must be a member of matcher.BindConflict, or None for the default strategy (ACCEPT).
on_merge – A merge strategy. Must be a member of matcher.BindMerge, or None for the default strategy (KEEP_LAST).

class Rebind(**kwargs)¶

Change the binding settings for all bindings in a submatcher.

For example, one might want bindings in one part of the AST matcher to merge with each other, but then want it to be an error if these conflict anywhere else.

Parameters

submatcher – The matcher whose bindings to rewrite.
on_conflict – A conflict resolution strategy. Must be a member of matcher.BindConflict, or None if on_conflict is not to be changed.
on_merge – A merge strategy. Must be a member of matcher.BindMerge, or None if on_merge is not to be changed.

class StringMatch(**kwargs)¶

Creates a synthetic string match.

For example, Bind("foo", StringMatch("asd"))` will result in being able to use $foo in substitutions, where it will be replaced with asd.

Since this produces a simple string match, it can only be used inside of non-syntactic templates, such as ShTemplate.

Python Data Structure Matchers¶

class Equals(**kwargs)¶: Matches a candidate iff it equals value.

class TypeIs(**kwargs)¶

Matches a candidate iff its type is precisely _type.

For example, TypeIs(ast.Expr) is roughly equivalent to the empty ast_matchers.Expr().

(This does not check any type information for the code that this candidate AST node might represent.)

class Contains(**kwargs)¶

Matches a collection if any item matches the given matcher.

Fails the match if the candidate is not iterable.

class HasItem(**kwargs)¶

Matches a container iff submatcher matches container[index].

Fails the match if the container doesn’t contain the key, or if the candidate node is not a container at all.

class ItemsAre(**kwargs)¶

Matches a sequence with an exact set of elements.

The matched sequence must have exactly the same number of elements (support len()) and each element will be matched against the corresponding matcher in matchers.

For example, this will create a matcher to match [1, 2], with a=1 and b=2:

>>> m = ItemsAre([Bind('a'), Bind('b')])

File Content Matchers¶

class MatchesRegex(**kwargs)¶

Matches a candidate iff it matches the regex.

The match must be complete – the regex must match the full AST, not just a substring of it. (i.e. this has re.fullmatch semantics.)

Any named groups are added to the bindings – e.g. (xyz) does not add anything to the bindings, but (?P<name>xyz) will bind name to the subspan 'xyz'.

The bound matches are neither lexical nor syntactic, but purely on codepoint spans.

class FileMatchesRegex(**kwargs)¶: Matches iff regex matches anywhere in the candidate’s file.

`syntax_matchers`¶

High level pattern matchers on AST nodes.

These should be preferred to ast_matchers, as they are safer and less brittle.

These are also available on the command line:

Syntax¶

Syntax patterns use $-prefixed words as metavariables, where that variable can stand for any syntax tree. For example, $x + 3 matches any addition operation where the right hand side is 3.

Additional restrictions can be placed on the tree using the restrictions parameter. For example, restrictions={'x': ExprPattern('4')} specifies that $x is not a wildcard, but rather, only matches an ExprPattern(4).

Warning

Patterns will match _exactly_ the same AST – down to the order of members in a set. These patterns are still useful tools, but should be combined with other matchers to match exactly what you want.

>>> list(matcher.find_iter(
...     ExprPattern('{1, 2}'),
...     matcher.parse_ast('{2, 1}')))  # fails
[]

Metavariables¶

Metavariables can be reused in a pattern, which constrains them to match an equivalent AST (BindConflict.MERGE_EQUIVALENT_AST).

For example, [$x for $x in $y] will match [a for a in b], but not [a1 for a2 in b].

For convenience, these are rebound to normal Bind() variables outside of the pattern matcher.

ExprPattern¶

class ExprPattern(**kwargs)¶

An AST matcher for a pattern expression.

ExprPattern creates a matcher that exactly matches a given AST, but also allows placeholders. For example, this will match any addition of two variables named literally foo and bar:

ExprPattern('foo + bar')

But this will match any addition expression at all:

ExprPattern('$foo + $bar')

In addition, whatever expressions $foo or $bar matched will be a bound variable in the match (under ‘foo’ and ‘bar’).

Parameters

pattern – The pattern to match, an expression.
restrictions – (optional) A dict mapping metavariables to matchers.

StmtPattern¶

class StmtPattern(**kwargs)¶

An AST matcher for a pattern statement.

StmtPattern is like ExprPattern, but for an entire (single) statement!

Like ExprPattern, StmtPattern creates a matcher that exactly matches a given AST, but also allows placeholders. For example, this will match any assignment where the literal variable foo is set to the literal variable bar:

StmtPattern('foo = bar')

But this will match any assignment statement at all:

StmtPattern('$foo = $bar'}

Parameters

pattern – The pattern to match, a statement.
restrictions – (optional) A dict mapping metavariables to matchers.

Tree-walking Matchers¶

class HasParent(**kwargs)¶

Matches an AST node if its direct parent matches the submatcher.

An AST node in this context is considered to be an AST object or a list object. Only direct parents are yielded – the exact object x s.t. the candidate is x.y or x[y], for some y. There is no recursive traversal of any kind.

Fails the match if the candidate node is not an AST object or list.

class IsOrHasAncestor(**kwargs)¶

Matches a candidate if it or any ancestor matches the submatcher.

If the candidate directly matches, then that match is returned. Otherwise, the candidate is recursively traversed using HasParent until a match is found.

class HasAncestor(**kwargs)¶

Matches an AST node if any ancestor matches the submatcher.

This is equivalent to HasParent(IsOrHasAncestor(...)).

class HasChild(**kwargs)¶

Matches an AST node if a direct child matches the submatcher.

An AST node in this context is considered to be an AST object or a list object. Only direct children are yielded – AST.member or list[index]. There is no recursive traversal of any kind.

Fails the match if the candidate node is not an AST object or list.

class IsOrHasDescendant(**kwargs)¶

Matches a candidate if it or any descendant matches the submatcher.

If the candidate directly matches, then that match is returned. Otherwise, the candidate is recursively traversed using HasChild until a match is found.

class HasDescendant(**kwargs)¶

Matches an AST node if any descendant matches the submatcher.

This is equivalent to HasChild(IsOrHasDescendant(...)).

class HasFirstAncestor(first_ancestor, also_matches)¶

The first ancestor to match first_ancestor also matches also_matches.

For example, “the function that I am currently in is a generator function” is a matcher that one might want to create, and can be created using HasFirstAncestor.

class HasPrevSibling(**kwargs)¶: Matches a node if the immediate prior sibling in the node list matches submatcher.

class HasNextSibling(**kwargs)¶: Matches a node if the immediate next sibling in the node list matches submatcher.

High Level Syntax Matchers¶

class NamedFunctionDefinition(**kwargs)¶

A matcher for a named function definition.

This includes both regular functions and async functions.

Parameters

body – The matcher for the function body.
returns – The matcher for the return type annotation.

class InNamedFunction(**kwargs)¶: Matches anything directly inside of a function that matches submatcher.

class WithTopLevelImport(**kwargs)¶

Matches an AST node if there is a top level import for the given module.

Parameters

submatcher – The matcher to filter results from.
module_name – The fully-qualified module name as a string. e.g. 'os.path'.
as_name – The variable name the module is imported as. Defaults to the name one would get from e.g. from os import path.

`lexical_matchers`¶

lexical_matchers provides lexical tweaks and filters on lexical matches.

class HasComments(**kwargs)¶

Filter results to only those lexical spans that have comments inside.

Parameters: submatcher – A Matcher matching a LexicalMatch.

class NoComments(**kwargs)¶

Filter results to only those lexical spans that have no comments inside.

Parameters: submatcher – A Matcher matching a LexicalMatch.

`ast_matchers`¶

Automatically generated low-level AST node matchers.

For each AST node in the ast module, there is a matcher with the same name, which accepts submatchers for each of its attributes.

For example, if the Python grammar has an entry like:

UnaryOp(unaryop op, expr operand)

Then the following matcher will match any ast.UnaryOp:

ast_matchers.UnaryOp()

And this will match any ast.UnaryOp with an op attribute matching submatcher1, and an operand attribute matching submatcher2:

ast_matchers.UnaryOp(op=submatcher1, operand=submatcher2)

(See the unit tests for more examples.)

class Subscript(**kwargs)¶

class Num(**kwargs)¶

class Bytes(**kwargs)¶

class Str(**kwargs)¶

class NameConstant(**kwargs)¶

class Ellipsis(**kwargs)¶

class AST(**kwargs)¶

class Add(**kwargs)¶

class And(**kwargs)¶

class AnnAssign(**kwargs)¶

class Assert(**kwargs)¶

class Assign(**kwargs)¶

class AsyncFor(**kwargs)¶

class AsyncFunctionDef(**kwargs)¶

class AsyncWith(**kwargs)¶

class Attribute(**kwargs)¶

class AugAssign(**kwargs)¶

class AugLoad(**kwargs)¶

class AugStore(**kwargs)¶

class Await(**kwargs)¶

class BinOp(**kwargs)¶

class BitAnd(**kwargs)¶

class BitOr(**kwargs)¶

class BitXor(**kwargs)¶

class BoolOp(**kwargs)¶

class Break(**kwargs)¶

class Call(**kwargs)¶

class ClassDef(**kwargs)¶

class Compare(**kwargs)¶

class Constant(**kwargs)¶

class Continue(**kwargs)¶

class Del(**kwargs)¶

class Delete(**kwargs)¶

class Dict(**kwargs)¶

class DictComp(**kwargs)¶

class Div(**kwargs)¶

class Eq(**kwargs)¶

class ExceptHandler(**kwargs)¶

class Expr(**kwargs)¶

class Expression(**kwargs)¶

class ExtSlice(**kwargs)¶

class FloorDiv(**kwargs)¶

class For(**kwargs)¶

class FormattedValue(**kwargs)¶

class FunctionDef(**kwargs)¶

class GeneratorExp(**kwargs)¶

class Global(**kwargs)¶

class Gt(**kwargs)¶

class GtE(**kwargs)¶

class If(**kwargs)¶

class IfExp(**kwargs)¶

class Import(**kwargs)¶

class ImportFrom(**kwargs)¶

class In(**kwargs)¶

class Index(**kwargs)¶

class Interactive(**kwargs)¶

class Invert(**kwargs)¶

class Is(**kwargs)¶

class IsNot(**kwargs)¶

class JoinedStr(**kwargs)¶

class LShift(**kwargs)¶

class Lambda(**kwargs)¶

class List(**kwargs)¶

class ListComp(**kwargs)¶

class Load(**kwargs)¶

class Lt(**kwargs)¶

class LtE(**kwargs)¶

class MatMult(**kwargs)¶

class Mod(**kwargs)¶

class Module(**kwargs)¶

class Mult(**kwargs)¶

class Name(**kwargs)¶

class Nonlocal(**kwargs)¶

class Not(**kwargs)¶

class NotEq(**kwargs)¶

class NotIn(**kwargs)¶

class Or(**kwargs)¶

class Param(**kwargs)¶

class Pass(**kwargs)¶

class Pow(**kwargs)¶

class RShift(**kwargs)¶

class Raise(**kwargs)¶

class Return(**kwargs)¶

class Set(**kwargs)¶

class SetComp(**kwargs)¶

class Slice(**kwargs)¶

class Starred(**kwargs)¶

class Store(**kwargs)¶

class Sub(**kwargs)¶

class Suite(**kwargs)¶

class Try(**kwargs)¶

class Tuple(**kwargs)¶

class UAdd(**kwargs)¶

class USub(**kwargs)¶

class UnaryOp(**kwargs)¶

class While(**kwargs)¶

class With(**kwargs)¶

class Yield(**kwargs)¶

class YieldFrom(**kwargs)¶

class alias(**kwargs)¶

class arg(**kwargs)¶

class arguments(**kwargs)¶

class boolop(**kwargs)¶

class cmpop(**kwargs)¶

class comprehension(**kwargs)¶

class excepthandler(**kwargs)¶

class expr(**kwargs)¶

class expr_context(**kwargs)¶

class keyword(**kwargs)¶

class mod(**kwargs)¶

class operator(**kwargs)¶

class slice(**kwargs)¶

class stmt(**kwargs)¶

class unaryop(**kwargs)¶

class withitem(**kwargs)¶

`extern_matchers`¶

Matchers for integrating with external tooling.

class RewriteFile(metavariable_prefix: str)¶: Base class for whole-file rewrites.

class ExternalCommand(metavariable_prefix: str, command: Union[str, Sequence[str]], shell: bool = False)¶: Runs an external command to modify a file.

refex.python.matchers¶

base_matchers¶

Control Flow Matchers¶

Examples¶

Binding Manipulation Matchers¶

Python Data Structure Matchers¶

File Content Matchers¶

syntax_matchers¶

Syntax¶

Metavariables¶

ExprPattern¶

StmtPattern¶

Tree-walking Matchers¶

High Level Syntax Matchers¶

lexical_matchers¶

ast_matchers¶

extern_matchers¶

`refex.python.matchers`¶

`base_matchers`¶

`syntax_matchers`¶

`lexical_matchers`¶

`ast_matchers`¶

`extern_matchers`¶