SqlTokenizer

API reference › yii › db › SqlTokenizer

abstract class	yii\db\SqlTokenizer
Extends	yii\base\Component » yii\base\BaseObject
Implements	yii\base\Configurable
Since	2.0.13
Source	framework/db/SqlTokenizer.php

1 subclass

• yii\db\sqlite\SqlTokenizer

SqlTokenizer splits an SQL query into individual SQL tokens.

It can be used to obtain an addition information from an SQL code.

Usage example:

$tokenizer = new SqlTokenizer("SELECT * FROM user WHERE id = 1");
$root = $tokeinzer->tokenize();
$sqlTokens = $root->getChildren();

Tokens are instances of SqlToken.

Properties

Property	Type	Description	Defined by
$behaviors read-only	array<yii\base\Behavior<static>>	List of behaviors attached to this component.	yii\base\Component
$sql	string	SQL code. Default: null.	SqlTokenizer
$length protected	int	SQL code string length. Default: null.	SqlTokenizer
$offset protected	int	SQL code string current offset. Default: null.	SqlTokenizer

Methods

Method	Description	Defined by
__construct()	Constructor.	SqlTokenizer
__call()	Calls the named method which is not a class method.	yii\base\Component
__clone()	This method is called after the object is created by cloning an existing one.	yii\base\Component
__get()	Returns the value of a component property.	yii\base\Component
__isset()	Checks if a property is set, i.e. defined and not null.	yii\base\Component
__set()	Sets the value of a component property.	yii\base\Component
__unset()	Sets a component property to be null.	yii\base\Component
attachBehavior()	Attaches a behavior to this component.	yii\base\Component
attachBehaviors()	Attaches a list of behaviors to the component.	yii\base\Component
behaviors()	Returns a list of behaviors that this component should behave as.	yii\base\Component
canGetProperty()	Returns a value indicating whether a property can be read.	yii\base\Component
canSetProperty()	Returns a value indicating whether a property can be set.	yii\base\Component
className() static	Returns the fully qualified name of this class.	yii\base\BaseObject
detachBehavior()	Detaches a behavior from the component.	yii\base\Component
detachBehaviors()	Detaches all behaviors from the component.	yii\base\Component
ensureBehaviors()	Makes sure that the behaviors declared in behaviors() are attached to this component.	yii\base\Component
getBehavior()	Returns the named behavior object.	yii\base\Component
getBehaviors()	Returns all behaviors attached to this component.	yii\base\Component
hasEventHandlers()	Returns a value indicating whether there is any handler attached to the named event.	yii\base\Component
hasMethod()	Returns a value indicating whether a method is defined.	yii\base\Component
hasProperty()	Returns a value indicating whether a property is defined for this component.	yii\base\Component
init()	Initializes the object.	yii\base\BaseObject
off()	Detaches an existing event handler from this component.	yii\base\Component
on()	Attaches an event handler to an event.	yii\base\Component
tokenize()	Tokenizes and returns a code type token.	SqlTokenizer
trigger()	Triggers an event.	yii\base\Component
indexAfter() protected	Returns an index after the given string in the SQL code starting with the specified offset.	SqlTokenizer
isComment() protected abstract	Returns whether there’s a commentary at the current offset.	SqlTokenizer
isIdentifier() protected abstract	Returns whether there’s an identifier at the current offset.	SqlTokenizer
isKeyword() protected abstract	Returns whether the given string is a keyword.	SqlTokenizer
isOperator() protected abstract	Returns whether there’s an operator at the current offset.	SqlTokenizer
isStringLiteral() protected abstract	Returns whether there’s a string literal at the current offset.	SqlTokenizer
isWhitespace() protected abstract	Returns whether there’s a whitespace at the current offset.	SqlTokenizer
startsWithAnyLongest() protected	Returns whether the longest common prefix equals to the SQL code of the same length at the current offset.	SqlTokenizer
substring() protected	Returns a string of the given length starting with the specified offset.	SqlTokenizer

Method details

__construct()

public function __construct(string $sql, array $config = [])

Constructor.

Parameters

Parameter	Type	Description
$sql	string	SQL code to be tokenized.
$config	array	Name-value pairs that will be used to initialize the object properties

tokenize()

public function tokenize(): yii\db\SqlToken

Tokenizes and returns a code type token.

Returns

SqlToken: Code type token.

indexAfter()

protected function indexAfter(string $string, int|null $offset = null): int

Returns an index after the given string in the SQL code starting with the specified offset.

Parameters

Parameter	Type	Description
$string	string	String to be found.
$offset	int|null	SQL code offset, defaults to current if null is passed.

Returns

int: Index after the given string or end of string index.

isComment()

abstract protected function isComment(int &$length): bool

Returns whether there’s a commentary at the current offset.

If this methos returns true, it has to set the $length parameter to the length of the matched string.

Parameters

Parameter	Type	Description
$length	int	Length of the matched string.

Returns

bool: Whether there’s a commentary at the current offset.

isIdentifier()

abstract protected function isIdentifier(int &$length, string &$content): bool

Returns whether there’s an identifier at the current offset.

If this methos returns true, it has to set the $length parameter to the length of the matched string. It may also set $content to a string that will be used as a token content.

Parameters

Parameter	Type	Description
$length	int	Length of the matched string.
$content	string	Optional content instead of the matched string.

Returns

bool: Whether there’s an identifier at the current offset.

isKeyword()

abstract protected function isKeyword(string $string, string &$content): bool

Returns whether the given string is a keyword.

The method may set $content to a string that will be used as a token content.

Parameters

Parameter	Type	Description
$string	string	String to be matched.
$content	string	Optional content instead of the matched string.

Returns

bool: Whether the given string is a keyword.

isOperator()

abstract protected function isOperator(int &$length, string &$content): bool

Returns whether there’s an operator at the current offset.

If this methos returns true, it has to set the $length parameter to the length of the matched string. It may also set $content to a string that will be used as a token content.

Parameters

Parameter	Type	Description
$length	int	Length of the matched string.
$content	string	Optional content instead of the matched string.

Returns

bool: Whether there’s an operator at the current offset.

isStringLiteral()

abstract protected function isStringLiteral(int &$length, string &$content): bool

Returns whether there’s a string literal at the current offset.

If this methos returns true, it has to set the $length parameter to the length of the matched string. It may also set $content to a string that will be used as a token content.

Parameters

Parameter	Type	Description
$length	int	Length of the matched string.
$content	string	Optional content instead of the matched string.

Returns

bool: Whether there’s a string literal at the current offset.

isWhitespace()

abstract protected function isWhitespace(int &$length): bool

Returns whether there’s a whitespace at the current offset.

If this methos returns true, it has to set the $length parameter to the length of the matched string.

Parameters

Parameter	Type	Description
$length	int	Length of the matched string.

Returns

bool: Whether there’s a whitespace at the current offset.

startsWithAnyLongest()

protected function startsWithAnyLongest(array<int, string>|array<int, array<string, mixed>> &$with, bool $caseSensitive, int|null &$length = null, string|null &$content = null): bool

Returns whether the longest common prefix equals to the SQL code of the same length at the current offset.

Parameters

Parameter	Type	Description
$with	array<int, string>|array<int, array<string, mixed>>	Strings to be tested. The method will modify this parameter to speed up lookups.
$caseSensitive	bool	Whether to perform a case sensitive comparison.
$length	int|null	Length of the matched string.
$content	string|null	Matched string.

Returns

bool: Whether a match is found.

substring()

protected function substring(int $length, bool $caseSensitive = true, int|null $offset = null): string

Returns a string of the given length starting with the specified offset.

Parameters

Parameter	Type	Description
$length	int	String length to be returned.
$caseSensitive	bool	If it’s false, the string will be uppercased.
$offset	int|null	SQL code offset, defaults to current if null is passed.

Returns

string: Result string, it may be empty if there’s nothing to return.