1 Introduction
~~~~~~~~~~~~
For most problems you need to identify the relevant parts of a
connection to trace. To do this consider which scenario you are
having problems with and where tracing needs to be enabled.
Note that tracing produces a lot of output , especially at higher
trace levels.
There are 3 main areas of Net that can produce trace output:
1 = the SQL*Net 'client'
2 = the 'listener' process
3 = the SQL*Net 'server'.
a) Establishing a connection:
Client ----> Listener ----> Server
^ ^ ^
1 2 3
b) An established connection:
Client --------> Server
^ ^
1 3
c) Opening a database link:
Client ----> Server ----> Listener -----> Server2
^ ^ ^ ^ ^
1 3 * 2 3
* Note here that the Oracle server process is also a SQL*Net
client when it makes an outgoing call to a listener to
open a database link. Database links are OPENED when first
used. They should then remain open until closed.
It is the 'server' trace which traces the outbound DBLINK
traffic.
d) An established database link:
Client ----> Server -----> Server2
^ ^ ^ ^
1 3 * 3
In each case here there are several potential sampling points. You
should be able to identify quickly which of these scenarios matches
your setup. As these scenarios are likely to involve connections
between different machines you should remember that tracing for any
process is controlled by the configuration details that the process
reads WHEN IT IS STARTED. This is especially important when looking
at MTS connections as the SQL*Net server is the 'dispatcher' process.
Some dispatchers are started when the database instance is started
and others may start at a later time (on demand). Each dispatcher will
read their SQL*Net configuration WHEN THEY START.
It is also important to note that to trace 'Server' processes started
by the listener then the 'sqlnet.ora' file used depends on the
environment passed to the 'Server' by the listener.
2 Client Tracing
~~~~~~~~~~~~~~
For client TOOLS edit or create the file $HOME/.sqlnet.ora and add
the lines:
trace_level_client=16
trace_file_client=cli
trace_directory_client=/tmp # Or a known directory
trace_unique_client=true # Add '_pid' to trace filename
This will turn on FULL tracing for your user account only producing
output in a file called /tmp/cli_
(For some SQL*Net versions the file will be just /tmp/cli.trc)
For client 'ORACLE' process (as in the case of database links) put this
same information into $TNS_ADMIN/sqlnet.ora file.
On versions up to and including Oracle 7.0.16 client trace may not
add a process ID to the name of the trace file. This means two
processes may end up writing to the same trace file unless you
take care to control which processes write trace output to each file.
3 Listener Tracing
~~~~~~~~~~~~~~~~
Listener tracing can ONLY be configured in the listener.ora file.
Add the lines below to the listener.ora file:
trace_level_listener=16
trace_file_listener=listener
trace_directory_listener=/tmp # Or a known directory
This will define FULL listener tracing to the file /tmp/listener.trc.
You can enable this tracing by either:
lsnrctl reload
OR
lsnrctl stop;
lsnrctl start;
TCP/IP
~~~~~~
It is often useful to confirm that a listener is listening on a
specified address. Most Unix machines include a command called
'netstat' (Often in /etc or in /usr/etc). The command netstat -a
should list all TCP/IP end points on which a listener is listening.
Eg:
For a listener listening on HOST=... PORT=1580 there should be a
netstat entry of the form:
RecvQ SendQ Local Address Foreign Address TCP state
0 0 *.1580 *.* LISTEN
Note: Some versions of netstat will only list established connections
and not listen end points. See the man page on your machine.
4 Server Tracing
~~~~~~~~~~~~~~
Server side trace is not required as often as the other two traces
mainly because most problems are related to establishing a connection.
Once a connection has been established the client and server processes
are communicating. It is sometimes useful to see exactly what SQL
commands have been received by the server, and what data it has sent
back out.
The file $TNS_ADMIN/sqlnet.ora controls the server side tracing. Add
the lines below to this file:
trace_level_server=16
trace_file_server=server
trace_directory_server=/tmp # Or a known directory
Output should be sent to the file /tmp/server_
Note: Server side tracing acts on the SQL*Net server side.
For dedicated connections this is the Oracle process on the
server machine.
For MTS connections this is the DISPATCHER and NOT the shared
server. Data is passed between the dispatcher and the shared
servers via the SGA and this does NOT involve SQL*Net.
It is also important to note that as a dispatcher handles
several client processes the dispatcher trace output can be a
mix of trace from many client processes making it VERY difficult
to follow. The general advice for such problems is:
a) See if the problem reproduces WITHOUT using MTS - if
so the trace is much cleaner
b) If a problem ONLY reproduces under MTS ensure the machine
is in a controlled environment so you can be sure that only
YOUR process is using the dispatcher.
5 Trace Summary
~~~~~~~~~~~~~
1) Identify where you need to trace.
2) Identify which files on which machines control tracing at these
points. Tracing is controlled in the following files:
Client Server Listener
~~~~~~ ~~~~~~ ~~~~~~~~
Files: $HOME/.sqlnet.ora sqlnet.ora listener.ora
sqlnet.ora
3) Add in the relevant trace parameters (See Below)
4) Restart any processes that need to read the new trace values.
Reload the listener as required.
5) Reproduce your problem
6) Save all your trace output immediately
7) Disable the tracing
6 Main Trace Parameters
~~~~~~~~~~~~~~~~~~~~~
trace_level_[listenerclientserver] = [offuseradmin0-16]
trace_file_[listenerclientserver] = Filename *1
trace_directory_[listenerclientserver] = Directory *2
*1 Unquoted (") filenames will be translated into lower case.
*2 You CANNOT use environment variables in the Filename or Directory
name.
7 Diagnosing Trace output
~~~~~~~~~~~~~~~~~~~~~~~
Trace output can be very difficult to follow. Before looking at a
trace file make sure:
a) You are familiar with the sequence of events in setting up
a connection. SQL*Net connections follow a sequence of
events - you will need to determin where in the sequence
the problem occurs.
b) Do not be misled by error reports in the trace files. You
must follow the context of the errors - an error may be
quite valid at that point in a sequence. Eg: For client
connections a list of addresses to call is built - if the
first address yeilds no response the next address is tried.
This next address may yeild a response and the 'true' error
occurs at this point in the sequence.
c) Do not be misled by unusual 'Bequeath' connections in the
trace. If an error is received over SQL*Net the client
may use a "Bequeath" operation to spawn an oracle process
which it then uses to get the TEXT of the error. A very short
exchange of packets occurs and the bequethed process exits.
The 'TRUE' problem is likely to be before this bequeath
operation.
Useful trace 'tags':
The following are useful items to follow in trace files - these
are not guaranteed to be valid across all SQL*Net releases and
are for guidance only. Entries are assumed to be taken at trace
level 16 to allow data packets to be seen. This will produce a
LOT of trace output.
-
Error information follows. Remember the error may be acceptable
osntns: Calling address
Shows address list constructed for a call OUT to a listener
nricall: Making call with following address information: ...
Shows the ACTUAL address being called from the above list
nsopen: entry
We are about to try and open a connection.
nsopen: transport is open
nsopen: error exit
A connection to the called address has been made / failed.
nsclose: ...
An established connection is being closed - check nearby
for errors.
nscall: redirected
The client has been redirected to a differenct address.
The next step should be to call the new address. The address
should appear in an earlier data packet.
nspsend / nsprecv
Outgoung / Incoming data
No comments:
Post a Comment