Friday, July 20, 2012

Working With history


Most Shells Provide Powerful mechanism for handling History that helps you to recall commands or repeat them. This is very helpful when executing a long command.

The History command available in the Bash shell shows you the available commands executed until now much like, 

[root@vx111a ~]#  history
  989  cd domains/
  990  ll
  991  clear
  992  cd foo_domain/
  994  cat fileRealm.properties
  999  ./startWebLogic.sh
 1000  exit
 1001  history 10
*
*

We can get a specific number of commands using, history 20 (which gives you last 20 commands in the history).The commands that we normally execute are saved in the file /root/.bash_history.

History Env variables

HISTFILE: There is an env variable called ‘HISTFILE’ which is pointed to the /root/.bash_history file making it to save commands executed. If we unset this env variable, the commands are not saved when shell exited.we can change this by

Export HISTFILE=/root/.commandline_warrior

HISTSIZE: This is another env variable specifying the number of commands to be remembered by the command history. The default size is 500. Just export the variable as

export HISTSIZE= 3000

HISTTIMEFORMAT: This env variable enabled to add time stamp to the associated command executed. Just export the variable as

export HISTTIMEFORMAT = ‘%F %T ‘

HISTCONTROL : When working at the command line we often end up executing some commands multiple times. The default history size is 500, too many duplicates of the same commands can fill up your history and leave you with a less then useful history. We can use this env variable to make that duplicate commands are not saved in history. This can done by adding

export HISTCONTROL= ignoredups

But the above env remove duplicate command if they are consecutive. If we need to erase the duplicated completely all over the history add,

export HISTCONTROL=erasedups

We can also force history not to remember particular commands .This is done by adding

export HISTCONTROL=ignorespace

At this point, when you enter a command with a ‘space’ before it. The command gets executed but it is not saved in history file.

If we need to make history ignore specific commands, we can

export HISTIGNORE="pwd:ls:ls -ltr:"

This will make sure that the commands are not allowed to execute.

Clean a History

If we need to clean the history completely, we can use ‘history –c’

Write & Read

We can make sure that our history can be written to a specific file ( besides /root/.bash_history). This can done using

history -w hist.txt

And Reading history from a file is also possible by

history -r hist.txt

Here is a list of examples that can be working on the bash shell along with history command

     
     !!

Execute the Previous Command

   
     !n

Execute the Nth command

!1000 : execute the command number 1000 which is saved in the history file

     
     !$


Take the arguments from the previous command

[root@vx111a test]# cat sam1
THIS IS JAGADISH1
[root@vx111a test]# nl !$ 
nl sam1
     1  THIS IS JAGADISH1

   
      !:0

Take the Previous Command

[root@vx111a test]# cat sam1
THIS IS JAGADISH1
[root@vx111a test]# !:0 sam2
cat sam2
THIS IS PAVAN


     !^ ( Or ) ^

Take the First argument

[root@vx111a test]# cat sam1
THIS IS JAGADISH1
[root@vx111a test]# nl !^
nl sam1
     1  THIS IS JAGADISH1

     
     !:n

Take the arguments from the last command. !:1 take the first argument , !:2 second and so on

[root@vx111a test]# cat sam1
THIS IS JAGADISH1
[root@vx111a test]# nl !:1
nl sam1
     1  THIS IS JAGADISH1
[root@vx111a test]# mv sam1 tam1
[root@vx111a test]# cat !:2
cat tam1
THIS IS JAGADISH1

    
    !:-

Previous Command
[root@vx111a test]# cat sam1 
THIS IS JAGADISH1
[root@vx111a test]# !:- sam1
cat sam1
THIS IS JAGADISH1

    
    !$

Takes the last Argument

[root@vx111a test]# mv tam1 sam1
[root@vx111a test]# nl !$
nl sam1
     1  THIS IS JAGADISH1

    
   !*

Take the Previous command argument

[root@vx111a test]# nl sam1
     1  THIS IS JAGADISH1
[root@vx111a test]# cat !*
cat sam1
THIS IS JAGADISH1

  
    !!:g


Replace the Occurrences (replaces all 1 with 2)

cat file1 file2 file3
!!:gs/1/2 ( when you execute this , all 1 will be replaced to 2 ) ,so the command would be cat file2 file2 file3


Happy Learning J
Read More

Thursday, July 19, 2012

Connecting To Weblogic JVM



Working as an Application System admin, I need to work very closely with Weblogic and JBoss. Since JBoss runs mostly on JDK versions, Weblogic has one more jdk available called JRocket.

 Most of the production servers contains Weblogic will be running on JRocket jvm since it provides some benefits that normal jdk. In this article we will see how we can communicate with Weblogic serer that started using JRocket jvm (thinking that the Weblogic is running on JRocket Jvm).

There is a tool called “jrcmd” provided with JRocket jdk which sends commands to the JRocket jvm. This is a command line tool available in the JRcoket/bin/.Considering the process ID for Weblogic is 3448.Here are a list of commands used with “jrcmd” for communicating with the Weblogic process.

1. Find the Version of the JRocket Running  

jrcmd 3448 version
3448:
Oracle JRockit(R) build R28.2.3-13-149708-1.6.0_31-20120327-1522-linux-ia32, compiled mode
GC mode: Garbage collection optimized for throughput, strategy: genparpar

2. Find the available Weblogic process running

jrcmd -p
4536 jrockit.tools.jrcmd.JrCmd -p
3448 weblogic.Server

3. Find the Commands provided by this version

jrcmd 3448 help
3448:
The following commands are available:
        kill_management_server
        start_management_server
        print_object_summary
        memleakserver
        print_class_summary
        print_codeblocks
        dump_codelayout
        dump_codelist
        dump_codemap
        print_codegenlist
        exception_trace_filter
        print_vm_state
        print_utf8pool
        check_flightrecording
        dump_flightrecording
        stop_flightrecording
        start_flightrecording
        print_properties
        hprofdump
        print_threads
        datadump_request
        runsystemgc
        runfinalization
        heap_diagnostics
        oom_diagnostics
        print_exceptions
        version
        timestamp
        command_line
        sanity
        verbosity
        set_filename
        help
        print_memusage
        set_vmflag
        list_vmflags
For more information about a specific command use 'help '.
Parameters to commands are optional unless otherwise stated.

If you need to find more about a command we can use, 

jrcmd 3448 help runsystemgc
3448:
Calls java.lang.System.gc().
        full        - do a full GC (same as -XXfullsystemgc). (bool, false)
        fullcompact - force a full compaction (only for full GCs). (bool,
                      true)

4. Find Out the Current Time Stamp and Up time of the VM

jrcmd 3448 timestamp
3448:
==== Timestamp ====  uptime: 0 days, 01:35:06 time: Tue Jul 17 14:36:25 2012


5. Find out the Command line and other information used to start the VM Instance

jrcmd 3448  print_vm_state
3448:
Uptime       : 0 days, 01:30:57 on Tue Jul 17 14:32:16 2012
Version      : Oracle JRockit(R) R28.2.3-13-149708-1.6.0_31-20120327-1522-linux-ia32
CPU          : Intel Core 2 SSE SSE2 SSE3 SSSE3 SSE4.1 Core Intel64
Number CPUs  : 2
Tot Phys Mem : 3179118592 (3031 MB)
OS version   : Red Hat Enterprise Linux Server release 5.6 (Tikanga)
Linux version 2.6.18-238.el5 (mockbuild@ls20-bc2-13.build.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-50)) #1 SMP Sun Dec 19 14:24:47 EST 2010 (i686)
.
.
.
.

6. Find out the Command line of a VM Instance

jrcmd 3448  command_line
3448:
Command Line: -Xms512m -Xmx512m -Dweblogic.Name=AdminServer Djava.security.policy=/soa/wls10/Middleware/wlserver_10.3/server/lib/
weblogic.policy Dweblogic.ProductionModeEnabled=true -da -Dplatform.home=/soa/wls10/Middleware/wlserver_10.3 -Dwls.home=/soa/wls10/Middleware/wlserver_10.3/server Dweblogic.home=/soa/wls10/Middleware/wlserver_10.3/server Dweblogic.management.discover=true -Dwlw.iterativeDev=false -Dwlw.testConsole=false -Dwlw.logErrorsToConsole=false Dweblogic.ext.dirs=/soa/wls10/Middleware/patch_wls1036/profiles/default/
sysext_manifest_classpath:/soa/wls10/Middleware/patch_oepe180/profiles/
default/sysext_manifest_classpath:/soa/wls10/Middleware/patch_ocp371/
profiles/default/sysext_manifest_classpath:/soa/wls10/Middleware/
patch_adfr1111/profiles/default/sysext_manifest_classpath -Dsun.java.command=weblogic.Server Dsun.java.launcher=SUN_STANDARD weblogic.Server


7. Find out the Properties available to the VM Instance

jrcmd 3448  print_properties
3448:

=== Initial Java properties: ===
java.vm.specification.name=Java Virtual Machine Specification
java.vm.vendor.url.bug=http://download.oracle.com/docs/cd/E15289_01/
go2troubleshooting.html
weblogic.home=/soa/wls10/Middleware/wlserver_10.3/server
wlw.iterativeDev=false
java.home=/soa/jrockit-jdk1.6.0/jre
java.vm.vendor.url=http://www.oracle.com/
weblogic.ext.dirs=/soa/wls10/Middleware/patch_wls1036/profiles/default/
sysext_manifest_classpath:/soa/wls10/Middleware/patch_oepe180/profiles/
default/sysext_manifest_classpath:/soa/wls10/Middleware/patch_ocp371/
profiles/default/sysext_manifest_classpath:/soa/wls10/Middleware/
patch_adfr1111/profiles/default/sysext_manifest_classpath
java.security.policy=/soa/wls10/Middleware/wlserver_10.3/server/lib/
weblogic.policy
wlw.testConsole=false
java.vm.specification.version=1.0
.
.
.


8. Find Out the memory Details 

jrcmd 3448 print_memusage scale=M baseline 
3448:
Total mapped                       773MB           (reserved=0MB)
-              Java heap            512MB           (reserved=0MB)
-              GC tables             17MB          
-          Thread stacks           15MB           (#threads=36)
-          Compiled code           8MB           (used=7MB)
-               Internal                0MB          
-                     OS                72MB          
-                  Other               73MB          
-        Java class data            72MB           (malloced=72MB #102209 in 18503 classes)
- Native memory tracking       1MB           (malloced=0MB #8)



9. Find out the Object Details

jrcmd 3448 print_object_summary
3448:

--------- Detailed Heap Statistics: ---------
29.9% 10418k   106949 +10418k [C
 7.8% 2703k   115352  +2703k java/lang/String
 5.8% 2023k    18501  +2023k java/lang/Class
 4.6% 1614k    13769  +1614k [Ljava/lang/Object;
 4.0% 1388k    15207  +1388k [Ljava/util/HashMap$Entry;
 3.8% 1326k     6590  +1326k [B
 3.5% 1227k     1059  +1227k [J
 2.5% 877k    37435   +877k java/util/HashMap$Entry
 2.5% 877k    28076   +877k java/util/LinkedHashMap$Entry
 1.6% 541k    17328   +541k java/util/concurrent/ConcurrentHashMap$Segment
 1.5% 522k     3347   +522k [I
 1.3% 446k     5398   +446k [Ljava/lang/String;
 1.2% 432k    18444   +432k java/util/Hashtable$Entry
 1.2% 425k     9085   +425k java/util/HashMap
 1.2% 407k    17385   +407k java/util/concurrent/locks/ReentrantLock$NonfairSync
 1.2% 403k    17236   +403k javax/xml/namespace/QName
 1.1% 387k     8267   +387k java/lang/ref/SoftReference
 1.1% 370k    17328   +370k [Ljava/util/concurrent/ConcurrentHashMap$HashEntry;
 0.9% 326k     5974   +326k java/util/LinkedHashMap
 0.9% 324k     5938   +324k com/bea/staxb/buildtime/internal/bts/QNameProperty
 0.9% 313k     8037   +313k java/lang/ref/WeakReference
 0.9% 311k     4443   +311k [Ljava/util/Hashtable$Entry;
 0.8% 278k    17817   +278k com/bea/staxb/buildtime/internal/bts/MethodName
 0.8% 276k     3538   +276k java/lang/reflect/Method
     34866kB total ---

--------- End of Detailed Heap Statistics ---

10. Find Information Regarding Loaded Class Details 

jrcmd 3448 print_class_summary

**class sun/jkernel/DownloadManager$2$2
**class sun/jkernel/DownloadManager$2$1
*class jrockit/vm/RNI
*class jrockit/vm/Reflect
*class jrockit/vm/VM
**class jrockit/vm/Locks
*class jrockit/vm/AccessController
*class java/lang/reflect/AccessibleObject
**class java/lang/reflect/Constructor
**class java/lang/reflect/Field
**class java/lang/reflect/Method
*
*
*

11. Run Garbage Collection

jrcmd 3448 help runsystemgc
3448:
Calls java.lang.System.gc().
        full        - do a full GC (same as -XXfullsystemgc). (bool, false)
        fullcompact - force a full compaction (only for full GCs). (bool,
                      true)


12. Run Finalization

jrcmd 3448 help runfinalization
3448:
Calls java.lang.System.runFinalization().

13. Generate a Thread Dump

jrcmd 3448 print_threads



14. Heap_diagnostics 

This can used in getting detailed information about the heap usage. When this command is executed a full GC is trigged during which the information is collected. The output normally contains 3 sections,

General Information: available memory and heap usage

Detailed heap Details : list the amount of heap that a particular class occupy, the amount of memory in Kbs that they consume, the total number of instances alive and other more information

Reference object statistics – lists the amount of total instances, how many instances are reachable, how many instances are unreachable and thus eligible for garbage collection, how many reference objects were found unreachable during this garbage collection, how many objects were activated before this garbage collection, how many instances are pointing to null and the instance types.

We can run the command like, jrcmd 3448 heap_diagnostics > /logs/heap.conf
If we are interested in a septic class, we can use

jrcmd heap_diagnostics | grep '%.*GenericClassLoader : This will grep for the one line we are interested in. The output seems to be

0.0% 10k 150 +10k weblogic/utils/classloaders/GenericClassLoader

15. Oom_diagnostics 

This allows to print the current state of heap when an Out of memory occurs. We can add the following variables to the startup files like

jrcmd 3448 oom_diagnostics > /logs/oos.conf

More to Come, happy learning 



Read More

Tuesday, July 17, 2012

Exploring the /etc/fstab


Fstab file is one of the most important file in linux. The main use is that it maps devices to locations so that devices can be used. It contains information of all the partitions and storage devices in your computer. We need to have root privileges for modifying the file.

A basic structure and some contents of the file will look like this,



Device
/ root  
/home
swap    
mount_location       
/
/home
swap
fileSystem
ext3
ext3
ext3

Options
default
rw,suid,dev,noexec,auto,nouser,async 
default
Dump
1
1
0
Fsck
1
2
0


Device is the underlying device

Mount location, is nothing but a location where we mounted the file system for accessing.

File System defines the type of the current file system such as ext3, fat, ext2, ntfs, swap e.t.c.There is a Special option available called auto which simply means that the file system type is detected automatically.

Options: these are options available for a partition. This is the confusing part in the file. If we know some of the options and what they actually do, it is pretty easy.

auto and noauto With the auto option, the device will be mounted automatically at boot up and this is the default options. If you don't want the device to be mounted automatically, we can use noauto. When we use noauto it will not be available until explicitly mounted.
ro Mount the filesystem read-only.
rw Mount the filesystem read-write
defaults Uses the default options that are rw, suid, dev, exec, auto, nouser, and async
exec and noexec exec lets you execute binaries that are on that partition, whereas noexec doesn't let you do that. noexec might be useful for a partition that contains binaries you don't want to execute on your system, or that can't even be executed on your system

These are not only the available ones. There are some more options available.

Dump : the dump is nothing but a linux utility for taking a  backup.When started dump checks the entry and uses the number to decide if a file system should be backedup or not

O: dump will ignore the file system (most cases)
1: dump will back up

Fsck : this reads the number in identifies in which order the file system check should be done.
0: will not check
1: root
2: other file systems

Happy learning J
Read More

Package Management


Previously Linux programs used to come in source code with all files, configuration stuff so that the user has to build them to work. Now days they are coming in the form of packages which ship ready for installation. In this article we will see the available package management tools in linux. Package management system is a collection of tools to automate the process of installing, upgrading, configuring, and removing software packages from a computer

Red Hat Package Manager (RPM)

RPM is a package management system used for packaging in the linux standard base. The rpm command options are grouped into three sub groups for ,

Querying and verifying packages
Installing, upgrading and removing packages
Performing miscellaneous functions

Normally rpm is the command available for performing the above operations and a rpm file will normally contain an extension of .rpm.

An RPM is delivered in a single file, normally in the format:

'name'-'version'-'release'.'architecture'.rpm

such as: libgnomeuimm-2.0-2.0.0-3.i386.rpm

Some rpm refer to package which do not depend on certain artchitecture.These includes graphics, text for another program to use.

The Rpm contents also include a package label , which contains the following information ,

  • the software name
  • the software version
  • the package release (the number of times the package has been rebuilt using the same version of the software)
  •  the architecture the package was built for (i386, i686) 
The package label does not necessarily need to match the name of the file.

Libraries are distributed in two separate packages for each version. One contains the precompiled code for use at run-time, while the second one contains the related development files such as headers, etc. Those packages have "-devel" appended to their name field. The system administrator should ensure that the versions of the binary and development packages match.

The Basic syntax for the rpm command would be,

Rpm

Where options include i (install), v (verbose), h (hash check), e (erase), u (Update), U (Upgrade)

Some Examples of rpm include,

 To install a RPM package
rpm -ivh httpd-1.0-2.i386.rpm
 To uninstall a RPM package
rpm -e httpd
To upgrade a RPM package
rpm -Uvh httpd-1.0-2.i386.rpm
To query a RPM package
rpm -q httpd
Verify an Installed Package
rpm --verify mysql
List Files in Installed Package
rpm -ql httpd
Find Out Which Package this file belongs
rpm -qf /etc/httpd/conf/httpd.conf
Find Out the Docs that came with the package
rpm -qd httpd
Learn about a package before installing
rpm -qip zsh-html-4.2.6-5.el5.i386.rpm
Learn What files a package contains
rpm -qa | grep httpd
Find all packages that match a specific pattern
rpm -qa | grep httpd
To verify whether a package is installed or not
rpm -V zsh-html-4.2.6-5.el5.i386.rpm
 To check a RPM signature package
rpm  --checksig httpd



YUM: Yellow Dog Update Modifier

There were some issues when we use rpm as a package management tool.Rpm cannot handle the dependencies. For this reason yum was released.YUM is said to be a high level wrapper for the rpm tool

A Local repository can be set up with all the packages. It can be seen here.

Some Examples of yum include,

 To install a package
Yum install httpd-1.0-2.i386.rpm
 To uninstall a package
Yum remove httpd
To update a package
Yum update httpd-1.0-2.i386.rpm
Display list of updates
Yum list updates
List all the installed packages
Yum list installed
Find information about package
Yum info httpd
Find for a specified package
Yum list installed | grep httpd
Search for a package name
Yum list httpd
Search for packages
Yum search httpd
Display list of available packages
Yum list all
Display list of grouped software
Yum grouplist
Install packages by group
Yum groupinstall pack (Install all packages that are under the group pack)
Remove all packages in a group
Yum groupremove pack
Find what packages provides a file
Yum whatprovides /etc/passwd
Find what packages need to be updated
Yum check-update


Yum also provides a interactive shell, called yum-shell which is invoked by ‘yum shell’. A Shell will be open which can allow multiple yum commands to be executed during a single executing of yum.

More To Come , Happy Learning J
Read More