I NTRODUCTION
DbAccess is an X-windows application developed in IDL that streamlines interactions with the NSTX Microsoft SQL database, offering a user-friendly interface tailored to the statistical and graphical requirements of NSTX physicists It supports flexible views and joins, enabling complex SQL expressions akin to "expression tables" found in other software The application incorporates GA Plot Objects, enhancing its graphical and interactive features with minimal programming effort Additionally, it simplifies the process of performing multiple linear least squares fits, with or without powers, and is compatible with the PPPL Linux cluster, such as the portal.
S ETTING U P D B A CCESS
To use DbAccess at PPPL you must:
To access NSTX databases on the PPPL Linux Cluster, enter the command `module load nstx/mdsplus` in the Linux command line after logging in, or include it in your `.cshrc` or `.bashrc` file This command configures IDL, MDSplus, and sets the necessary environmental variables for Sybase access If you are a new user, please refer to the NSTX user setup documentation for guidance.
To obtain a database account, you need to contact the database administrator by emailing dbadmin@pppl.gov Upon the creation of your account, you will also receive a corresponding database password.
3) To optionally see some buttons in color, when using Unix or MacOS X as your X- window manager, you need the following lines in your ~/.Xresources file:
Idl*colorbuttons*blue*background:blue
Idl*colorbuttons*red*background: red
Idl*colorbuttons*lightblue*background: lightblue
Idl*colorbuttons*green*background: green
Idl*colorbuttons*purple*background: purple
Idl*colorbuttons*yellow*background: yellow
Idl*colorbuttons*white*background: white
Idl*colorbuttons*gold*background: gold
Idl*colorbuttons*black*background: black
Idl*colorbuttons*magenta*background: magenta
Idl*colorbuttons*orange*background: orange
To copy the entries on a PPPL Unix host, use the command `cat /p/nstxusr1/util/init_files/idlcolors.Xresources >> ~/.Xresources` Alternatively, when using dbaccess, you can select "Load Button Color Resource" from the "File" menu.
To access the NSTX database, you must create a file named .sybase_login in your home directory for each database you intend to use Currently, the primary database for NSTX data is nstxlogs Consequently, your nstxlogs.sybase_login file should contain the following details: sqlnstx:8080 sqlnstx nstxlogs.
The example above requires an MDSplus server running on the database server Leave the first line blank, and this won’t be necessary.
To create the necessary files on a PPPL Unix host, execute the script located at /p/nstxusr1/util/setup/overwritedbfiles.sh, which will generate this file and others if they are not already present If you intend to use different databases, you will need to manually create the corresponding files.
S TARTING D B A CCESS
Type IDL at the command prompt Once inside IDL type dbaccess. portal% idl
Upon launching the application, users will see a list of tables and defined views in the default nstxlogs database, categorized under “Database Tables.” By selecting any table or view, the corresponding columns will be displayed under “Column Names.” For additional assistance, users can click the “Help” button located at the top right to access this document or seek further support.
Simple Queries
A simple query retrieves all records from a single table For querying data from multiple tables or a specific subset of a table, refer to the relevant section.
“Complex Queries.” To select all the data in a particular table:
To view the columns of a specific table, click on its name in the "Database Tables" column, which will then display the corresponding columns in the "Column Names" section.
2) Select all of the columns you wish to view in the column labeled “Column Names” with your mouse
3) Then click the red “>>” button located to the right of the “Column Names” so that the selected columns appear in the column labeled “Work Area.”
4) Click the yellow “Select” button.
A new window will appear containing a table with the information you have selected.
Complex Queries
i) Using Functions and Arithmetic Operators
SQL databases use "Select" statements to retrieve records, allowing for the inclusion of various functions and arithmetic operators For example, a valid Select statement could be "Select LOG(Ip), ABS(Bt/1000) from xp19."
These functions perform a calculation, usually based on input values provided as arguments, and return a numeric value.
To use a function or arithmetic operator:
To view the columns of a specific table, simply click on its name in the “Database Tables” column This action will display the corresponding columns in the “Column Names” section.
2) Select all of the columns you wish to view in the column labeled “Column
Names” with your mouse (either one at a time, or in groups; shift-clicking and control-clicking should work).
3) Then click the red “>>” button located to the right of the “Column Names” causing the selected columns appear to in the column labeled “Work Area.”
4) Double-click on the column name in the work area You will see an Edit window appear.
5) Enter the function and any necessary parameters with parentheses around the column name.
6) Click “Done.” You will see that the changes you have made to the column value are shown in the work area
7) You may now click the “Select” button, “Plot” button, etc.
Instead of double-clicking to add a function, you can simply drag the desired item from "Column Names" to the "Work Area" by pulling down to the appropriate line using the F( )> button.
In the table returned to you from the “Select” button, you should see a column labeled with the function you entered in the work area: ii) Complex query without a join
To execute a complex query without a join:
To view the columns of a specific table, click on its name in the "Database Tables" column, which will then display the corresponding columns in the "Column Names" section.
2) Select all of the columns you wish to view in the column labeled “Column
Names” with your mouse (shift-clicking and control-clicking should work)
To add selected columns to the "Work Area," click the red ">>" button next to "Column Names." You can incorporate functions and mathematical operators by double-clicking the line or by dragging the desired item from the F( )> button into the "Work Area."
4) Once the desired columns are in the “Work Area,” choose “Constrain Data Set” from the Edit menu
5) Select “Use a single table select statement” (the default option) and click “OK.”
A new window titled "DbAccess Constrain Data Set" will open, displaying the items from the "Work Area" in the first column If no specific items are selected, all will be included in the query results The second column features drop-down boxes containing all items from the "Work Area," allowing you to limit the results to specific criteria.
6) Select the column you wish to limit from the drop down list (E.g., neut_day.shot)
7) Then select a Boolean from the third column (E.g., “>”)
8) Enter a value in the 4 th column (E.g., 107000)
Completing these steps with the example values will result in the following select statement:
Select neut_day.day, neut_day.neut,neut_day.shot from neut_day where neut_day.shot >107000
9) Select “And” or “Or” from the last column (e.g., “And”).
10) Select a column from the second drop-down box (e.g., select neut_day.shot again)
11) Select a Boolean from the drop-down in the 3 rd column (e.g., “107000 and neut_day.shot < 109000
To view your requested results, click the yellow "Select" button at the bottom of the window This action will close the "DbAccess Constrain Data Set" window and display a new window with the results For instance, the example provided will retrieve all records from the neut_day table where shot values are between 107000 and 109000, not including the endpoints.
To limit a column that is a string data type, such as logbook comments in the
“Entries” table or username in the logbook you must use SINGLE QUOTES This indicates to IDL that the value you are entering is a string.
The SQL query retrieves logbook entries that meet three criteria: the shot value exceeds 109,000, the entry is made by Dennis Mueller, and the comment includes the term "fiducial," which may be surrounded by any number of characters.
Select ENTRIES.SHOT, ENTRIES.TEXT, ENTRIES.USERNAME from ENTRIES where ENTRIES.SHOT>109000 And ENTRIES.USERNAME='mueller' And ENTRIES.TEXT Like '%fiducial%' iii) Complex query with a join
Creating a query with a join allows you to select data from multiple tables based on shared criteria A typical example is retrieving records from two tables where the shot numbers match However, it's important to note that queries involving joins can be significantly slower than those that query a single table.
To execute a complex query with a join:
To view the columns of a specific table, simply click on its name in the "Database Tables" column This action will display the corresponding columns in the "Column Names" section.
2) Select all of the columns you wish to view in the column labeled “Column Names” with your mouse (shift-clicking and control-clicking should work).
To add selected columns to the "Work Area," click the red “>>” button next to the “Column Names” column Just like in the earlier section on “Using Functions and Arithmetic Operators,” you can incorporate functions and mathematical operators with any items in the “Work Area.”
4) Repeat the above steps for columns from another table.
5) Then chose “DbAccess Constrain Data Set” from the “Edit” menu
6) Select “Use a join statement” and click “OK.”
A new window titled "DbAccess Constrain Data Set" will open, displaying items from the "Work Area" of the main window, such as the Bt column from the xp19 table (xp19.bt) If you do not select specific items, all will be included in your query results Additionally, there is a section labeled "Join" for further configuration.
Criteria” each of the drop-down boxes contain all of the items you placed in the
“Work Area.” Additionally, all drop-down boxes in the section labeled “Where
To join the neut_day table with the entries table using the shot column, ensure that all items in the "Work Area" are included in the main program window.
7) In the section labeled “Join Criteria” select from the first drop-down box
8) Since we are performing an equa-join leave the “=” in the second drop-down box.
9) In the third drop-down box choose “neut_day.shot”
Clicking the yellow “Select” button at this point would result in the following statement being executed:
Select ENTRIES.SHOT, ENTRIES.TEXT, ENTRIES.XP, neut_day.day, neut_day.neut, neut_day.shot from ENTRIES JOIN neut_day ON
(ENTRIES.SHOT = neut_day.shot)
You can view your results, including the text, XP, day, and the number of neutrons per day for each shot, which are detailed in both the entries and neut_day tables.
Alternatively, this selection could be further limited by doing the following:
10) In the area labeled “Where Clause:” Select “Entries.shot.”
11) Then select the Boolean “>” from the second column.
12) Enter a value in the 3rd column (109000).
In the section for selecting columns to be returned, choose multiple columns while avoiding redundancy; for instance, there is no need to select both nuet_day.shot and entries.shot when joining on these columns, so only one should be selected.
After completing these steps, the following SQL statement would be executed:
Select ENTRIES.TEXT, ENTRIES.XP, neut_day.neut, neut_day.shot from ENTRIES JOIN neut_day ON (ENTRIES.SHOT = neut_day.shot) where ENTRIES.SHOT>109000
The following results would be returned: iv) Adding brackets for logical expressions
Finding Descriptions for a Particular Table
The "Description" table in the "nstxlogs" database is essential as it holds detailed information about each column, including MDSplus definitions, units, and textual descriptions of parameters For instance, to access the definitions of entries in the EFIT table, one can refer to this table for comprehensive insights.
To view the columns of the "Description" table, click on its name in the "Database Tables" section of the main program window, which will display the corresponding column names in the designated area.
2) Then click the red “>>” button located to the right of the “Column Names” so that these selected columns appear in the column labeled “Work Area.”
3) Then choose “DbAccess Constrain Data Set” from the “Edit” menu
4) Since we are only working with the “Description” table, select “Use a single table select statement” and click “OK.”
5) In the first drop-down box select “DESCRIPTION.Table_name.”
6) Leave the “=” in the second column.
7) Then put ‘Efit’ in the field as shown above.
8) Click the yellow “Select” button at the bottom of “DbAccess Constrain Data Set” window.
After completing these steps, the following SQL statement would be executed:
Select DESCRIPTION.Column_name, DESCRIPTION.Description,
DESCRIPTION.MDSplus, DESCRIPTION.Table_name, DESCRIPTION.Units from DESCRIPTION where DESCRIPTION.Table_name='Efit'
And the following results will be returned:
We strive to maintain this table with the most current information; however, if you cannot locate what you need, please contact us at dbadmin@pppl.gov for assistance.
Resizing the window
In DbAccess, resizing windows with the mouse automatically reorganizes the display for optimal viewing For instance, in the twm window manager, users can click and drag the rightmost square on the blue title tab to adjust the window size Additionally, column lines can be dragged to resize columns as needed.
Saving the Results
At the bottom of any window with returned results from a select statement you will notice that there is a yellow button at the bottom labeled “Save As.”
Clicking the designated button opens the "Save As" dialog, allowing you to select the file's location and name By saving your file as a csv (e.g., myfile.csv), you ensure it is stored in a comma-delimited format, making it compatible with Microsoft Excel for further analysis and use of standard spreadsheet features.
Printing the Results
C REATING A V IEW (E XPRESSION T ABLE )
A view in a database is akin to the Expression Tables from the older TFTR INGRES Database, allowing for flexibility in complexity It represents a set or subset of data from existing tables or views, with columns derived from various combinations of other tables, functions, and arithmetic operators, all encapsulated in a single SQL statement For instance, a practical example of a view statement can be found in the NSTX database.
CREATE VIEW dbo.haccess_charles AS SELECT shot, [time], DBkey, bt0, ip, 0.054 * POWER(nebar_ts * 1e-14, 49) * POWER(bt0, 85) *
POWER(psurfa, 84) AS pth, psurfa, rsurf, pnbi, prad, ptot, a, poh, r0, nebar_ts, 0.65 * POWER(nebar_ts * 1e-14, 93) * POWER(bt0, 86) *
POWER(rsurf, 2.15) AS pthmr, 10 * POWER(nebar_ts * 1e-14, 84) *
POWER(bt0,.63) * POWER(psurfa, 95) * POWER(a / rsurf, 46) AS pthaspect, 54 * POWER(nebar_ts * 1e-14, 49) * POWER(bt0, 85) *
POWER(psurfa, 84) AS pthsurfa, 65 * POWER(nebar_ts * 1e-14, 93) * POWER(bt0, 86) * POWER(rsurf, 2.15) AS pthmajor, 1.73 * POWER(nebar_ts
* 1e-14, 63) * POWER(bt0, 72) * POWER(a, 82) * POWER(rsurf, 99) AS pthryter, phase, wbdot, wpdot, 0.050 * POWER(nebar_ts * 1.0E-14, 0.46) * POWER(bt0, 0.87) * POWER(psurfa, 0.84) AS pth2, 1.67 * POWER(nebar_ts * 1.0E-14, 0.61) * POWER(bt0, 0.78) * POWER(a, 0.89) * POWER(rsurf, 0.94)
AS pthryter2, (pnbi + poh - wpdot) / 1E6 AS ploss FROM dbo.haccess
You can include a column in your view that matches one from another table, like "bt0" in the example provided Additionally, it's possible to create a column that is derived from a mathematical combination of multiple columns.
0.054 * POWER(nebar_ts * 1e-14, 49) * POWER(bt0, 85) *
In your view, the column "pth" is defined as a mathematical combination of the columns "nebar_ts," "bt0," and "psurfa" from the "haccess" table, utilizing the POWER function with a value of 84 for "psurfa."
The syntax "CREATE VIEW dbo.haccess_charles AS SELECT" illustrates that a view is essentially a saved database query Whenever data is retrieved from the view, the corresponding select statement is executed, ensuring that the view consistently reflects the most current information from the database.
The SQL statement used to define a view can incorporate data from multiple tables through join syntax, as well as limit the data retrieved from one or more tables by applying specific parameters.
Database”) To create your own view:
1) If you want to include a join in the statement for your join or you want your join to look at a subset of data from another table, bring the relevant column names into the
In the "Work Area" column of the main program window, you can incorporate various functions and mathematical operators as needed (refer to the section on "Using Functions and Arithmetic Operators") To proceed, simply click the yellow "Create View" button located on the right side of the main program window.
2) Using the example above, I have named my columns “pth” and “bt0” and put in specifies that the column “bt0” is in table “haccess.” This is important as there may be more then one column named “bt0” in the database.
3) You should see that your Username is already filled in for you You need to enter a name for your view in the field labeled “View Name.”
4) You can put in join criteria and column restraints as in the section “Selecting Data from the Database.”
5) Click the yellow “Create View” button at the bottom of the window.
6) Click the yellow “Refresh” button at the bottom of the main program window You should see that your view appears in the list of tables under “Database Tables.” You can now interact with this view as if it were a regular table.
C REATING A T ABLE
When creating a table for the first time, it's beneficial to familiarize yourself with the process and experiment with various scripts To do this effectively, consider setting up your table in a test database initially For detailed guidance, refer to the relevant section in the documentation.
“Working with a Different Database.” To create a new table:
1) Click the yellow “Create Table” button on the main program window This will cause a new “Create a Table” window to appear.
2) Enter a unique name for your new table in the field labeled “Table Name.”
3) You should see that your username is already in the field labeled “UserName of Table Creator.” In future versions of DbAccess this will be used to implement database security.
4) Enter names for the columns of your table in the column labeled “Column Name.”
5) Enter the SQL data type of each column in the column labeled “Data Type.” The choices are: char
Fixed-length non-Unicode character data can be defined with a length of n bytes, where n ranges from 1 to 8,000, resulting in a storage size of n bytes In SQL-92, the synonym for char is character, and it is recommended to use char when the data values in a column are anticipated to be consistently similar in size.
Variable-length non-Unicode character data can have a length ranging from 1 to 8,000 bytes, with the actual storage size reflecting the entered data's byte length rather than the maximum value of n It is possible for the data to be 0 characters long In SQL-92, the terms "char varying" and "character varying" serve as synonyms for varchar It is advisable to use varchar when the data values in a column are anticipated to vary significantly in size.
The date and time data spans from January 1, 1753, to December 31, 9999, with a precision of one three-hundredth of a second, which is approximately 3.33 milliseconds The values are rounded to increments of 000, 003, or 007 seconds For example, a timestamp may appear as 2002-06-13 08:08:07.080.
Integer (whole number) data from -2^31 (-2,147,483,648) through 2^31 - 1
(2,147,483,647) Storage size is 4 bytes The SQL-92 synonym for int is integer. float
A floating-point number can represent values ranging from -1.79E+308 to 1.79E+308 The precision and storage size of this number are determined by the number of bits, denoted as 'n', used for the mantissa in scientific notation, which can range from 1 to 53.
6) The “Length” column is optional If specified it will define the precision of the data type chosen in the previous step If not specified the default precision for each data type will be used:
Varchar 1 Datetime Cannot be set 23
7) In the “Allow Nulls” column you should specify with a Y (meaning this column will allow NULL values) or N (meaning this column will not allow NULL values) This is an important consideration especially if you will be loading your table by an automatic script (see section entitled “Populating a Table”) SQL will not create a record for you if you try to create one with an empty or unspecified value for a column that does not allow NULL values and this could result in no entry for that particular shot If you have a column named shot, for example, this column should likely not allow NULL values On the other hand, if it is possible that a particular value will not be able to be read from MDSplus for any given shot, it is safer to allow NULL values for that column The default is Y (allow NULL values) If you do not want a given column to allow NULL values you will need to change this to N for that column
8) In the “Units” column enter the units for the given column If there are no units for a column (such as shot number) enter “N/A.” This information will be stored automatically in the “Description Table.” (See section entitled “Finding Descriptions for a Particular Table”)
9) In the “Description” column enter a textual description of what your column will represent This is very useful for other people who may want to make use of your table, and will also serve as a good reminder for you This information will be stored automatically in the “Description Table,” as was the contents of the “Units” column
10) Click the yellow “Create Table” button at the bottom of the window If you have left any items blank you will receive an error Otherwise you should receive a message stating that your table has been created successfully
11) Click the yellow “Refresh” button on the main program window and you should see the addition of the table you have just created.
In the "Create Table" window, you can initially create a table with up to 20 rows and columns To add more columns, simply click the yellow "Add another column" button at the bottom, which will provide additional rows for defining extra columns.
Every table includes an additional column named "dbkey," which serves as the primary key, ensuring that it is unique and cannot contain NULL values This column is primarily utilized for housekeeping purposes and enables future modifications of table values through DbAccess Therefore, there is no need for you to create a separate primary key for your table.
From a File
To populate an already existing table from a file:
To create a data file for your database table, start by ensuring the first line matches the column headings of the table, regardless of their order Make sure the columns in your file are separated by tabs For instance, if you follow the example table from the previous section, your data file will be structured accordingly.
In the dataset, one IP value is missing, which is indicated by a space (" ") in the corresponding column, allowing the application to recognize it as empty Additionally, there are two entries for shot 109000; however, this will not create any issues since a unique key was automatically generated during table creation It's important to note that the "toi" char column was created without a specified length, resulting in a default length of just one character Consequently, when retrieving values from this table, only a single character will be displayed for each entry.
To modify the length of the "toi" column or to add or remove columns from your table, you must update the database structure accordingly.
To populate a table, choose your desired table from the “Database Tables” column in the main program window, where you will find a list of the columns included in that table.
Click the yellow “Populate Table” button on the right side of the main program window You will see the following dialog:
Choose “Populate manually from a file” and click “OK.”
In the "Load Data From a File" dialog, ensure that the Database Name and Table Name are displayed Input the file name and the path to the text file that contains your table data.
“File Name Inclucing Path:” field Click “OK”
Changing the database name in this dialog alters the database context, causing the application to search for the NewNstxTable in a different database This change will also impact the database context when returning to the main window You can verify this by clicking the “Refresh” button For instructions on switching back to a different database, refer to the section “Working with a Different Database.”
From a Script
M ODIFYING A T ABLE
Modifying a table within this application focuses on altering the table's structure rather than changing specific record values, which will be possible in future updates through DbAccess For instance, if you want to add a new column, change a column's data type, or adjust the units listed in the description table, you can accomplish these tasks using the "Modify" functionality in DbAccess To initiate the modification process, begin from the main program window.
1) Select the table you would like to modify from the “Database Tables” column Click the yellow “Modify” button on the right side of the main program window.
2) You will see a new “Modify Table” window appear and in it you will see the relevant information for the table you have chosen, which is read from the database.
3) Using as an example the table we created in the section entitled “Creating a Table,” you will see the data types and lengths of the columns you previously defined As pointed out the length of the toi column is indeed 1 Now is a good time to change it to accommodate our full time of interest names Change the 1 in the Length column to 10 or 15 Being sure that some portion of the row for column “toi” is selected, click the “Modify Column” button.
4) Also here, you can change a column to disallow null values For instance, we will never want to have an entry in the database with a NULL shot number so we should change the Y to an N in the “Allow Nulls Y/N” column for shot Being sure that some portion of the row for column “shot” is selected, click the “Modify Column” button.
5) It is also possible to add new columns here by simply typing in the information for the new column below those already existing Make sure you have some part of the row containing the column you are adding selected and clicking the yellow “Add filled with NULL values for all previous records and you will need to take some course of action to fill that column for all previous records For this reason, if your table already contains data, it does not makes sense to add a new column with an “N” in the “Allow NULL Y/N” column and doing so will cause and error and the column addition will not take place Also, if you already have a script created to load your table on a shot-by-shot basis you will need to modify this script to accommodate the new column you have added.
P LOTTING
Clicking the Plot button in the Database Access Utility or DbAccess Selection window opens a dialog that allows you to configure plots using the variables listed in the Work Area column of the main widget.
In the analysis, plasma current (Ip) will be represented on one axis, distinguishing data sets for shots less than or equal to 105000 and those greater than or equal to 105000, each displayed in different colors and potentially varying line styles and plot symbols Additionally, the second plot frame will feature an overlay of Neutral Beam Power and OH Power across all shots.
GA Plot Objects, such as those found in ReviewPlus, facilitate graphical representations by providing an interactive plot window This window features radio buttons for selecting mouse operation modes and a check button for enabling or disabling cursor tracking Users can create a zoom box by clicking the left mouse button when the Zoom option is activated, while a middle button click restores autoscaling Additionally, activating the Mark button allows users to highlight the corresponding row in the data table by clicking on a point in the plot Right-clicking within the plot opens configuration options similar to SCOPE, enhancing user experience and functionality.
To change symbols, colors, character sizes, etc., of any data set, select “Set Plot
Appearance” under the edit menu This should bring up the “GA Plot Properties Dialog” window:
See http://web.gat.com/comp/analysis/uwpc/reviewplus/manual/ for more detail
You may wish to use a commercial PC or Macintosh product, such as Excel, for plotting (see the section on “Saving your Results”).
S TATISTICAL F EATURES
After moving the desired columns into the Work Area, click on the Select Button to bring up the DbAccess Selection window:
Click on the “Analyze” button and you should see:
To run the model, first select a Y value, such as xp19.taue, and choose the desired Effects by dragging the mouse over the remaining variables and clicking “> Add >.” Next, ensure the “Use Powers” option is checked before clicking on “Run Model.” This action will prompt the display of two windows with the results.
After you have run a model, the button “Dist of Y-Model” button will become active Clicking it will show you the distribution of points about the mean:
You can create this plot for individual parameters by clicking on a parameter in the
“DbAccess Model” window, and clicking the “