Python Tokens
Tokens or Lexical unit are the smallest buidling blocks of a program. Tokens are the equivalent of alphabets , grammer and tenses of a English language.
Python has the following tokens:-
Example:- False, True, for, while, None, break, if, elif, else
b. Identifiers(Names) - The names given to different parts of the program like variables, function, objects, classes etc. eg:-
a = 2, add(), here a and add are identifier names
Rules for framing identifier names.
Eg:- Area = 3.142 * r * r , here 3.142 is the constant.
Types of literals:-
a. String literals b. Numeric literals c. Boolean literals d. Specific literals e. Literal collections
1. String Literals - Text enclosed in double quotes or single quotes(" ") forms a string literals in python.
Example:- "Astha", "abc123", "1+9-23" , '12345'
Escape Sequence(Non graphic characters) - \a , \b, \v, \t
Types of Strings:-
a. Single-line Strings - The strings that you create by enclosing text in quotes(single and double)
Example:- name = "Raj Singh"
b. Multiline Strings - Text stored across multiple lines as one single string.
Multiline strings can be written using backslash(/) or triple quotation marks.
Example:- name = "raj\
singh"
name = '''raj
singh'''
2. Numeric Literals - Types of numeric literals:-
a. Integer Numbers - Whole number without any fractional part. It may contain + or - sign. Commas cannot appear in integer constant.
i. Decimal Integer Literals - It consists of a sequence of digits. example:- 1234, 41,-92.
ii. Octal Integer Literals - It consists of digits between 0 to 7 and starts with 0o. example:-0o10, 0o987 is invalid octal.
iii. Hexadecimal Integer Literals - It consists of digits between 0 to 9 and A - F. Starts with 0x or 0X Example: 0x1AC, 0x000.
b. Floating Point Literals - Also called real literals. example:- 2.34, -2355.43323
They can be written in two forms:-
a. Fractional Form - It consists of signed(+ and -) and unsigned(only +) digits including a decimal point between digits. Example:- 2.0,-0.00053
Invalid floating points - 7, 13/2, 43,332.322
b. Exponent Form - A real literal in exponent form consists of two parts: mantissa and exponent. Example:- 0.58E01 is 5.8, 163E5, 0.56E7, -0.45E4, 4.E2, 2.4E-2, 543.4E+2 are all valid.
Invalid floating points:- 1.65E, 0.44E2.4, 44,442E02
3. Boolean Literals - A boolean literal is either True or False.
4. Special Literal None -It is used to indicate the absence of a value. example:- a = None
Python has the following tokens:-
- Keywords
- Identifiers(Names)
- Literals
- Operators
- Punctuators
Example:- False, True, for, while, None, break, if, elif, else
b. Identifiers(Names) - The names given to different parts of the program like variables, function, objects, classes etc. eg:-
a = 2, add(), here a and add are identifier names
Rules for framing identifier names.
- Identifier name should be a combination of letters(a-z, A-Z) and digits(0-9).
eg:- Valid names - abc123, abc, xy_123, xy_
Invalid names - abc#12, age$ - First character must be a letter or underscore.
eg:- Valid names- _123, a_123
Invalid names - 1abc - Upper and lower case are different.
eg:- ABC and abc are both different identifiers
Xyz and xyz are also different and valid to use in same program. - Identifiers can be unlimited in length.
eg:- "abcfdfsfgfshfsdgfjgsdfhsweoir89r98324757364rebf23t45jfbe647egjfg45rufbmndbfuy..." is valid and can be even more than this. - Keywords can not be used as identifier name.
eg:- Invalid names - if , else, elif, while as all of them are keywords in python.
Valid names - if_123, whilexy. - Whitespace(space,tab) characters cannot be used in identifier names.
eg:- Valid names - "abc 123"
Eg:- Area = 3.142 * r * r , here 3.142 is the constant.
Types of literals:-
a. String literals b. Numeric literals c. Boolean literals d. Specific literals e. Literal collections
1. String Literals - Text enclosed in double quotes or single quotes(" ") forms a string literals in python.
Example:- "Astha", "abc123", "1+9-23" , '12345'
Escape Sequence(Non graphic characters) - \a , \b, \v, \t
Types of Strings:-
a. Single-line Strings - The strings that you create by enclosing text in quotes(single and double)
Example:- name = "Raj Singh"
b. Multiline Strings - Text stored across multiple lines as one single string.
Multiline strings can be written using backslash(/) or triple quotation marks.
Example:- name = "raj\
singh"
name = '''raj
singh'''
2. Numeric Literals - Types of numeric literals:-
a. Integer Numbers - Whole number without any fractional part. It may contain + or - sign. Commas cannot appear in integer constant.
i. Decimal Integer Literals - It consists of a sequence of digits. example:- 1234, 41,-92.
ii. Octal Integer Literals - It consists of digits between 0 to 7 and starts with 0o. example:-0o10, 0o987 is invalid octal.
iii. Hexadecimal Integer Literals - It consists of digits between 0 to 9 and A - F. Starts with 0x or 0X Example: 0x1AC, 0x000.
b. Floating Point Literals - Also called real literals. example:- 2.34, -2355.43323
They can be written in two forms:-
a. Fractional Form - It consists of signed(+ and -) and unsigned(only +) digits including a decimal point between digits. Example:- 2.0,-0.00053
Invalid floating points - 7, 13/2, 43,332.322
b. Exponent Form - A real literal in exponent form consists of two parts: mantissa and exponent. Example:- 0.58E01 is 5.8, 163E5, 0.56E7, -0.45E4, 4.E2, 2.4E-2, 543.4E+2 are all valid.
Invalid floating points:- 1.65E, 0.44E2.4, 44,442E02
3. Boolean Literals - A boolean literal is either True or False.
4. Special Literal None -It is used to indicate the absence of a value. example:- a = None
Comments
Post a Comment