[Cockcroft98] Section 17.1——17.4

Chapter 17. The SymbEL Language

SymbELis an interpretive language, based on C, that was created to addressthe need for simplified access to data residing in the SunOS kernel.Although kernel interface API libraries have made this data moreavailable to C application programmers, it is still out of reach bymeans other than writing and compiling a program. The goal was tocreate a language that did not need compilation to run and would allowthe development of scripts that could be distributed to a largeaudience via e-mail without sending very large files or requiring theend user to possess a compiler.

Thecontrol and data structures of SymbEL are extensive enough to allow acomplex program to be written, but the language is not burdened withsuperfluous syntax, and the resulting grammar is far more compact thanC. The original interpreter,se,was developed under SunOS 5.2 FCS-C and tested on an MP690, SC1000, andan LC. Since then, it has been modified and tested on every subsequentrelease of SunOS. The latest release is SunOS 5.6, a.k.a. Solaris 2.6.The SPARC and Intel platforms are supported, and every effort has beenmade to support new devices and configurations for each of thesehardware platforms. These interpreter does not now, nor will it ever, run under any release of SunOS 4.x.

Thischapter assumes that you are conversant in the C programming language.Expertise is not necessary, but you should be comfortable with C tobest understand this chapter.

Collection of Performance Data

Thefirst step in understanding the relevance of the collected performancedata is to become familiar with the interface from which this datacomes. Solaris has several such interfaces that need examination.

Tuning and Performance

Operatingsystem performance is one of the most painful areas of systemadministration because it seems to involve so much magic. In fact,gaining proficiency in performance tuning is much like any otherendeavor. Time and experience are the best teachers.

Yetfor all that is known about tuning, the need to tune systems becomesmore desperate because of the lack of adequate tuning tools. This lackof tools stems from the rapid advances in machine architecture andoperating system design. And as the underlying technology continues toexpand, the existing tools for performance tuning become insufficientto solve performance problems.

SunOSis an excellent starting place for the development of tools for tuning.The existing tools shipped with SunOS provide a wealth of dataregarding the workings of the kernel in its management of the processspaces and of the underlying hardware. It is from the examination ofthis data that the decisions on how to tune the system are made.

Traditionally,examination of this data is a manual process: typing in commands,examining the output, and making decisions based on some preset rules.This process is time consuming and tedious, especially when you try totalk another person through it over the telephone or explain theprocedure via e-mail. It also leaves the information about how to makedecisions regarding system performance where it does the least good—insomeone’s memory.

In anattempt to translate this expertise from memory to written form, onecould write C, Bourne, or Korn shell scripts to run these commands,extract the output, and compare them to known threshold values. Thisapproach serves the purpose of making the expertise more portable andthus more accessible to the masses. It also impacts the performance ofthe machine under analysis enough to render the script useless as a“round the clock” diagnostician. Access to the data is needed withoutthe overhead of the multiple applications and the shell language.

Thefirst step toward solving this problem is to determine how theapplications themselves retrieve the data from the kernel. Anapplication programming interface (API) can then be built that performsthe same actions as these applications. These APIs could extract theirdata from any of these sources on SunOS 5.x.

We examine these APIs:

• kvm — Kernel virtual memory

• kstat — Kernel statistics

• mib — Management information base

• ndd — Network device driver

Thekvm Library

Thekvm library is an API for reading the values of kernel variables from the kernel memory. Before thekvm library, this reading was all done manually, usingopen(),lseek(), andread(). The code for reading values from the kernel is repetitive, so thekvmlibrary was developed. It solved this problem and provided additionalfunctionality for reading core files and checking the validity of theaddresses being read. Thekvm library functions that perform the work arekvm_open(),kvm_nlist(),kvm_read(),kvm_write(), andkvm_close().

Thekstat Library

A limitation of thekvm library is that the program reading values from kernel memory must have superuser permission because/dev/kmemis not readable by the world. A solution developed by Sun is theaddition of a kernel framework that is accessible for read-only accessby the world and write access by the privileged user. This framework iscalled thekstat framework.kstatalso has the benefit of having low overhead and being extensible toaccommodate any new kernel statistics that may be useful on futurearchitectures.

Thekstat interface,/dev/kstat, is an entry point into thekstat driver. It resides in the kernel and collects data from predefined functional areas within various subsystems and drivers.kstatcollects data from I/O and VM subsystems and the HAT layer, and it alsogathers various information about network interfaces and CPU activity.

The software interface to/dev/kstat is thekstat library,libkstat. It contains functions for opening and reading thekstat data from thekstat driver and for the superuser, writing data back to the kernel.

When thekstatdata is copied out of the kernel, it is put into a linked list ofstructures. Each node in the list is a structure that contains dataabout a particular functional area and identifies the format of thedata. The possible data formats are listed below.

KSTAT_TYPE_RAW —The data is a pointer to a structure, commonly defined in a /usr/include/sys include file.

KSTAT_TYPE_NAMED— The data is an array of structures that represent a polymorphic type.Each structure contains a textual name, a type designation, and a unionof members of the supported types. The supported types are:

KSTAT_DATA_CHAR 8 bits signed or unsigned KSTAT_DATA_INT32 32 bits signed KSTAT_DATA_UINT32 32 bits unsigned KSTAT_DATA_INT64 64 bits signed KSTAT_DATA_UINT64 64 bits unsigned

KSTAT_TYPE_INTR —The data is a pointer to a structure containing an array of unsignedlong values whose indices represent the type of interrupt. The types of interrupts and their index values are

KSTAT_INTR_HARD=0 From the device itself KSTAT_INTR_SOFT=1 From the system software KSTAT_INTR_WATCHDOG=2 Timer expiration KSTAT_INTR_SPURIOUS=3 Entry point called with no interrupt pending KSTAT_INTR_MULTSVC=4 Interrupt serviced prior to return from other interrupt service

KSTAT_TYPE_IO —The data is a pointer to a structure containing information relevant to I/O.

KSTAT_TYPE_TIMER —The data is a pointer to a structure containing timing information for any type of event.

The structures used by thekstat library are defined in/usr/include/sys/kstat.h. Note that thekstatframework is constantly evolving, and developing code around it isprecarious because the mechanism may grow, shrink, or change at anytime.

Management Information Base (MIB)

Themanagement information base is a set of structures containing datarelative to the state of the networking subsystem of the kernel. Thenature, structure, and representation of this data are based on RFC1213, Management Information Base for Network Management of TCP/IP-based internets: MIB-II.Specifically, data is stored regarding the performance of the kerneldrivers for IP, TCP, UDP, ICMP, and IGMP. This data is contained in onestructure for each driver. You can retrieve the data by building astream containing the drivers whose MIB data is to be read, sending acontrol message downstream, and reading the results sent back by allmodules in the stream that interpreted the message and responded.

The structures are also declared as variables in the kernel address space and therefore can be read by conventionalkvm methods.

The Network Device Driver ( NDD) Interface

TheNDD interface is another source of data regarding the state of thenetworking subsystem. Whereas the MIB is a source of data that isstatistical in nature, the NDD interface allows for tuning of drivervariables that control the function of the driver. The NDD interfacecan access variables kept by the IP, TCP, UDP, ICMP, and ARP drivers.Whereas the interface to the MIB structures is through the streamscontrol message interface, retrieval of NDD variables is through a muchsimplerioctl() mechanism. The structures kept by the drivers are also accessible through thekvm interface but are rather convoluted and therefore are most easily modified by use of the NDD interface.

Summary

Fromthese sources comes a great deal of data to manage in an applicationwhen the mechanisms used to retrieve it are the conventional APIs. Itis possible to distill this information into library calls so that theinformation is more generalized. The next step is to develop asyntactic notation for accessing the data, a language whose mostsignificant feature is the retrieval of the necessary data from theappropriate places without the notational overhead of actuallyperforming the read. What follows is the description of a language thatwas designed to remove the complication of accessing these values andto provide a framework for building applications that can make use ofthese values. This language is called SymbEL, the Symbol EngineLanguage.

Basic Structure

Inresponse to the need for simplified access to data contained inside theSunOS kernel, the SymbEL (pronounced “symbol”) language was created andthe SymbEL interpreter,se, wasdeveloped. SymbEL resembles C visually and contains many similarsyntactic structures. This orthogonality was done for ease of use andto leverage existing programming knowledge.

First Program

Since the best way to learn a new language is through example, let’s look at an example SymbEL program.

main() 
{
  printf("hello, world\n"); 
}

Thisprogram shows that the language structure is similar to C. Thissimilarity does not imply that you can pick up a book on C and startwriting SymbEL programs. There are a lot of differences, and theinterpreter lets you know if the syntax is incorrect. This shortprogram demonstrates some of the similarities and differences in thelanguage. First, a SymbEL program must have a function calledmain. This is the similarity to C. The difference from C is that theprintf() function in SymbEL is a built-in, not a library call.

To test this small program, put it in a file calledhello.se and enter

se hello.se

and the resulting output is what you expect. The format string used inprintf() works the same as any equivalent statement that could be used in a C program. Syntax of theprintf() format string is not addressed in this document; you can find additional information in theprintf() man page or in any good book on the C language.

Simple Types

The simple types available in SymbEL are scalar, floating point, and string.

Scalar

Floating Point

doubledouble precision floating point

String

stringpointer to null-terminated ASCII text

Thereare also structured types, which are covered later. There are nopointer types in SymbEL. This absence is also discussed in a latersection.

Simple Program

Thefollowing program demonstrates some of the features of the language,although it has nothing to do with extracting data from the kernel.

/* print the times table for 5 */ 
main() 
{
  // C++ style comments are also allowed 
  int lower; 
  int upper; 
  lower = 1;   // lower limit 
  upper = 10;  // upper limit 
  while(lower <= upper) {
    printf("%4d\n", lower * 5); 
    lower = lower + 1; 
  } 
}

About the Program

lower = 1;   // lower limit

Sincese uses the C preprocessor to read the programs, the use of C and C++ style comments and pound-sign directives such as#define and#include is valid.

while(lower <= upper) {

Thewhile structure is one of the control structures supported by SymbEL.

This example requires curly braces around the two statements inside the block of thewhileloop. However, the syntax of SymbEL requires all sequences ofstatements inside control structures to be bracketed by curly braces,even sequences of just one statement. The purpose for this requirementis to ensure cleanliness of the grammar.

printf("%4d\n", lower * 5);

The functionprintf()is a built-in function. Built-in functions are used for libraryfunctions that cannot be implemented as attached functions or thatrepresent a functionality specific to the interpreter. Attachedfunctions are discussed on page 532.

lower = lower + 1;

This code simply incrementslower by one.

Embellishing the Simple Program

Let’s rewrite the program, using different constructs and techniques.

#define LOWER 1 
#define UPPER 10 
#define DIGIT 5 
main() 
{
  int lower; 
  int upper = UPPER; 
  for(lower = LOWER; lower <= upper; lower = lower + 1) {
    printf("%4d\n", lower * DIGIT); 
  } 
}

About the Program

#define LOWER 1

Aspointed out before, the use of pound-sign directives is valid becausethe interpreter reads the program through the C preprocessor.

int upper = UPPER;

Variablescan be initialized to an expression. The expression can be anothervariable, a constant, or an arithmetic expression. Assignment of theresult of a logical expression is not allowed.

for(lower = LOWER; lower <= UPPER; lower = lower + 1) {
  printf("%4d\n", lower * DIGIT); 
}

The syntax of thefor loop in SymbEL will provide plenty of errors from the parser. It is a simpleforloop syntax insofar as it may only have one initial assignment and onedo-after statement. Multiple statements separated by commas won't work.Remember, the rule of thumb is “If you feel like being tricky, don't.It probably won't work.”

Thiscontrol structure is very convenient for many situations, but there arespecific rules that go with it. SymbEL does not allow the flexibilityof thefor loop that C does.

Array Types

SymbELsupports single-dimension arrays. Subscripts for array types start atzero, as is the case in C. The next example is a new version of theprevious example, using arrays.

A Program That Uses Arrays

#define LOWER 1 
#define UPPER 10 
#define DIGIT 5 
main() 
{
  int value[UPPER + 1]; 
  int i; 
  for(i = LOWER; i <= UPPER; i++) {
    value[i]=i*DIGIT; 
  } 
  for(i = LOWER; i <= UPPER; i++) {
    printf("%4d\n", value[i]); 
  } 
}

About the Program

int value[UPPER + 1]; 
for(i = LOWER; i <= UPPER; i++) {

The size of the array is declared asUPPER + 1.You can use expressions as the size of an array in its declarationprovided that the array size expression is an integral constantexpression. That is, it can’t contain other variables or function calls.

The size of the array must beUPPER + 1 because thefor loop uses the(i <= UPPER) condition for terminating the loop. If the array were declared with a size ofUPPER, thense would abort with a “subscript out of range” error during execution.

Also, instead of using thei=i+1 notation, we used the increment notation. Prefix and postfix increment and decrement are supported by SymbEL.

Functions and Parameters

So far, only one function has been shown in the examples:mainwith no return type. If no return type is declared on a function, thenreturning values will result in an error from the parser. A functioncan be declared as returning a type, and the function can then return avalue of that type. For example, here is a function to raise a value toa power.

A Program That Uses Functions and Parameters

#define BASE 2 
ulong 
power(int b, int p)        // raise base b to power p 
{
  ulong i; 
  ulong n; 
  n=1; 
  for(i=1; i<=p; i++) {
    n*=b; 
  } 
  return n; 
} 
main() 
{
  int i; 
  for(i=0; i<=32; i++) {
    printf("%d raised to power %d = %u\n", BASE, i, power(BASE, i)); 
  } 
}

About the Program

ulong 
power(int b, int p)        /* raise base b to power p */

This code is the declaration of a function returning unsignedlongand taking as parameters two integers. All parameters in SymbEL arevalue parameters and have the syntax of ANSI C parameters. As in localdeclarations, each parameter must be declared separately. The syntaxdoes not support comma-separated lists of parameters. Parameters aretreated as local variables and have all of the semantic constraints oflocal variables.

In this example, thepower() function is declared beforemain.This allows the parser to obtain type information about the function’sreturn value so that type checking can be done on the parameters toprintf().

int i; 
int n; 
n=1;

It is important to note a semantic feature of local variables that this code demonstrates. In C, the assignment ofncould be made part of the declaration. Although SymbEL supports theinitialization of local variables, the semantics of local variables areequivalent to astatic declaration in a Cprogram. That is, the initialization of the local variable is done onlyonce, upon entry to the function for the first time; it is notperformed on every call. The rationale is that the overhead ofmaintaining automatic variables in an interpretive environment would betoo high for the language to perform reasonably. One of the goals ofse is to put as little load on the system as possible and still provide usable runtime performance.

for(i=1; i<=p; i++) {
  n*=b; 
}

Theforloop is like the one shown in the previous example; however, thestatement inside the loop shows that SymbEL supports the compressedarithmetic assignment operators. These work just like the Ccounterparts and are supported for the operations add, subtract,multiply, divide, and modulus.

printf("%d raised to power %d = %u\n", BASE, i, power(BASE, i));

Here, the call topower() is part of the parameter list forprintf(). The order of evaluation of actual parameters is from left to right.

Global Variables

SymbELalso supports global variables. These act in the same way as globals inC. They are declared outside of the block of a function and can beaccessed by any function in the program. Global variables can haveinitial values just like local variables and are initialized before thefunctionmain is called.

Operators and Expressions

Tohelp you better understand what operators are available in SymbEL andhow expressions are constructed, here is an overview of the constituentparts.

Variable Names

Variablenames are limited to 1,024 characters and can begin with any alphabeticcharacter or an underscore. Characters beyond the first character canbe any alphabetic character, digit, underscore, or dollar sign.

Constants

Thereare four types of constants: integer, floating point, character, andstring. These constants can be used in expressions (except forstrings), assigned to variables, and passed as parameters. Forconvenience, these constants can be placed into#define statements for expansion by the preprocessor.

Integer

Aninteger constant can be any integer with or without sign whose value isno more than 2 to the 32nd power minus 1. Note that a large enoughvalue that has been negated results in sign extension. Constants oftypelonglong are not currently supported;a workaround is to add .0 to the end of the constant and assign theconstant to a variable of typedouble. Thedouble variable can then be assigned to a variable of typelonglong.

Floating Point

A floating-point constant can be

{0-9} [. {0-9}] [ {eE} {+-} {0-9}+ ]

Here are some examples of valid floating-point constants:

0.01

1e-2

1.0e-2

0.001E+1

These constants all represent the same value, 0.01.

Character

Acharacter constant is a character enclosed in single forward quotes. Asin C, the character may also be a backslash followed by anotherbackslash (for the backslash character itself) or a character thatrepresents a special ASCII value. The current special characters are:

Thevalue following the backslash may also be an octal or hex value. For anoctal value, use a backslash followed by a valid octal digit beginningwith 0. For hex, use a backslash followed by the characterx followed by a valid hex digit. Examples are:

String

Astring constant is enclosed in double quotes and can contain regularASCII characters or any of the special characters shown above. Examplesare:

"hello" 
"hello\nworld\n" 
"\thello world\xA"

A a special string value,nil, defines a pointer value of zero. String variables can be assignednil and compared to it. Parameters of typestring can also be sentnil as an actual parameter.

Declarations

Thesyntax for a variable declaration is the same for local and globalvariables. There may be an initialization part that is an expression.An oddity of the interpretive environment is that the initializationpart of a global declaration may contain a function call. Take greatcare not to break your program, however.

Some example declarations are

char c; 
intn=5; 
ulong ul = compute_it(n); 
string hello = "world";

Note that the syntax allows only one variable per line.

Arithmetic Operators

The current operators available for performing arithmetic on numeric values are

Theseoperators work the same as they do in C, and arithmetic operationsbetween numeric variables of different types are allowed. Explicitmodulus of floating-point types will be caught and disallowed by theparser; however, if the result of an expression yields a floating-pointvalue and is used as an operand to the modulus operator, it isconverted to typelonglong first, then performed. This feature has the potential for yielding unexpected results, so be wary when using it.

Inexpressions that contain mixed types of scalar or scalar and floatingpoint, the resulting expression will be of the type of the highestprecedence. The order of precedence from lowest to highest is:

char 
uchar 
short 
ushort 
int / long 
uint / ulong 
longlong 
ulonglong 
double

So, in this statement—

int fahrenheit; 
double celsius; 
... 
celsius = (5.0 / 9.0) * (fahrenheit - 32);

—the resulting value will bedouble. The(5.0 / 9.0) expression yields adouble, while(fahrenheit - 32) yields along (int islong internally). The multiply operator then changes the right side of the expression into adouble for multiplication. Take care not to lose accuracy because SymbEL does not support type casting.

The precedence of the operators+ and- is at the same level and of lower precedence than the*,/, and%operators, which are also at the same level. Parentheses can be used tocreate explicit precedence, as in the expression above. Withoutparentheses, the implicit value of the expression would be

celsius = (5.0 / (9.0 * fahrenheit)) - 32;

Logical Operators

The logical operators of SymbEL are

The logical operators are all of equal precedence because the syntax does not allow for expressions of the typea. The junctural operators for short circuit logical AND and short circuit logical OR are&& and||,respectively. They are implicitly of lower precedence than thecomparators. There is no logical NOT defined by the language. You canrewrite statements without the logical NOT operation by comparing thevalue to zero or by using inverse comparators and/or juncturals.

The regular expression comparison operator=~expects the regular expression on the right side of the operator.Comparisons with regular expressions on the left side of the operatorwill not yield the expected results.

Bitwise Operators

The bitwise operators of SymbEL are

for(i=0; i<10; i+=2) {
  ... 
}

int int_var = 5; 
ulong pointer = &int_var;

#include  
#include  
#include  

main() 
{
  char buf[128]; 
  iovec_t iov[1]; 

  iov[0].iov_base = &buf; 
  iov[0].iov_len = sizeof(buf); 
  if (readv(0, iov, 1) > 0) {
    fputs(((string) iov[0].iov_base), stdout); 
  } 
}

if (logical expression){
  ... 
}

if (running) {
  ... 
}

if (running != 0) {
  ... 
}

if(x>y){
  printf("x is greater than y\n"); 
} 
if(x  printf("x is less than y\n"); 
} else {
  printf("x is greater than y\n"); 
}

if (x == 2) {
  printf("x is equal to 2\n"); 
} else if (x == 3) {
  printf("x is equal to 3\n"); 
}

if (x == 2) {
  printf("x is equal to 2\n"); 
} else {
  if (x == 3) {
    printf("x is equal to 3\n"); 
  } 
}

main() 
{
  int i; 
  for(i=1; i<=10; i++) {
    printf("%2d %s\n", i, ((i % 2) == 0 ? "even" : "odd")); 
  } 
}

// character switch 
main() 
{
  char t = 'w'; 

  switch(t) {
  case 'a': 
    printf("a - wrong\n"); 
    break; 
  case 'b': 
    printf("b - wrong\n"); 
    break; 
  case 'c': 
    printf("c - wrong\n"); 
    break; 
  case 'w': 
    printf("w - correct\n"); 
    break; 
  default: 
    printf("say what?\n"); 
    break; 
  } 
} 
// string switch 
main() 
{
  string s = "hello"; 

  switch(s) {
  case "yo": 
  case "hey": 
  case "hi": 
    printf("yo/hey/hi - wrong\n"); 
    break; 
  case "hello": 
    printf("%s, world\n", s); 
    break; 
  default: 
    printf("say what?\n"); 
    break; 
  } 
}

    

while( logical expression ) {
  ... 
}

while(running) {
  ... 
}

while(running != 0) {
  ... 
}

for( assign part; while part; do part ) {
  ... 
}

i=0

i*=2

for(i = 0; i < 10; i++) {
  ... 
} 
for(i *= 2;i<10||x!=3;i=recompute(i)) {
  ... 
} 
for(; i < 10; ) {
  ... 
} 
for(;;) {
  ... 
}

do {
  ... 
} while( logical expression );

for(i=0; i<10; i++) {
  if (i == 2) {
    printf("i is 2\n"); 
    break; 
  } 
} 
printf("now, i is %d\n", i);

iis2 
now,iis2

while(i != 18) {
  if (i < 10) {
    continue; 
  } 
  i = do_something(); 
}