How do I list all available disks in Linux?

Sam Caldwell Atlassian Team Mar 14, 2015

Problem:

When building automation, sometimes we need to iterate through all of the available disks on the system.  What's a quick and effective way to do this?

2 answers

1 accepted

This widget could not be displayed.

man df

df -kh

Sam Caldwell Atlassian Team Mar 16, 2015

Matt, df is a great command. I use it a lot. But I'm not sure it works for this use case. For example, when I run it against one server, I get {code} scaldwell@reallysecretnode:~$ df -kh Filesystem Size Used Avail Use% Mounted on /dev/mapper/hidden--vg-root 00G 0.1G 14G 24% / udev 0.0G 0.0K 0.9G 1% /dev tmpfs 0.0G 000K 0.6G 1% /run none 0.0M 0 0.0M 0% /run/lock none 0.0G 0 0.9G 0% /run/shm /dev/mapper/something 0.0G 00M 0.0G 0% /tmp /dev/mapper/obfuscated 00G 00M 00G 0% /opt/obfuscated /dev/mapper/data 00G 00G 00G 00% /opt/j2ee/hidden /dev/mapper/obfuscated 00G 00M 00G 0% /opt/secret /dev/xda1 00M 00M 00M 00% /obfuscated {code} What I need in my particular case is to be able to separate disks from partitions and such. This is why I chose lsblk instead of df. Is there a right answer? Not really. I guess it depends on the specific user case. In either case, Matt, you have raised a great alternative. The next question is timing and performance. If we ignore use-case specifics and assume that df or lsblk are equivalent, then we would probably want to know which command runs faster. I did a quick test between meetings and found that df takes 0.003s on average. Here's where Matt's suggestion wins it: lsblk takes 0.005s to finish on average. It all comes down to use-case first then performance.

This widget could not be displayed.
Sam Caldwell Atlassian Team Mar 14, 2015

I was working on this problem today for a project.  Here's what I came up with...

for disk in $(lsblk -l | grep disk | awk '{print $1}'); do 
	#
	# Do work on the disk here.
	# 	
done

 

For example, in the case of my current project, I want to mount all of these partitions to /mnt/glusterfs/$disk...

for disk in $(lsblk -l | grep disk | awk '{print $1}'); do 
	mkdir -p /mnt/glusterfs/$disk && \
	mount /dev/$disk /mnt/glusterfs/$disk && \
	echo "$disk mounted successfully."	
done

Now the above exists in many places on the Internet, but let's add a twist to separate the rock stars from the newbies...

Assume you want to operate only on the disks that are NOT mounted to the system (because when we are deploying systems, we almost always have a mounted file system).  How do we filter out these already-mounted and healthy file systems?

for i in $(lsblk -l | grep disk | awk '{print $1}');do 
	if [ $(lsof | egrep $i[0-9]+ | wc -l) -eq 0 ]
	then
		echo "$i is not mounted."
	else
		echo "$i is mounted."
	fi
done

Now that's a little bit more awesomeness.  We know that because the file system is mounted, then a DIR (directory) and one or more files will be open.  Our friend `lsof` always tells us about open files, so testing the output of this command with a simple regex and counting the number of lines (wc -l) in the result gives us an idea as to whether or not a disk is in use.  That means we can do something like this to a virtual machine deployment we are automating....

for i in $(lsblk -l | grep disk | awk '{print $1}');do 
	[ $(lsof | egrep $i[0-9]+ | wc -l) -eq 0 ] && {
		echo "y" | mkfs.ext4 /dev/$i
        mkdir -p /dev/glusterfs/$i
        mount dev/$i /dev/glusterfs/$i
	}
done

In the above example, we only care about disks that are not in use.  We identify these disks and then use `mkfs.ext4` to format the disk with the ext4 file system.  Since we want an automated job, we echo a "y" character and pipe this to `mkfs.ext4` to avoid being prompted for any confirmation.

Have fun!

Sam Caldwell Atlassian Team Mar 14, 2015

Disclaimer: As a fellow Atlassian SOPS Engineer (ahem, thanks Harold) pointed out just now, `lsof` is expensive. Using lsof -n will save you a considerable amount of effort. However, using the `lsof` command in a loop should be avoided in most use cases. In the above examples, the expense of `lsof` is mitigated by the fact that it is to be used in a machine deployment scenario 10s after boot.

Suggest an answer

Log in or Sign up to answer
Atlassian Summit 2018

Meet the community IRL

Atlassian Summit is an excellent opportunity for in-person support, training, and networking.

Learn more
Community showcase
Posted Wednesday in Teamwork

What teamwork quotes inspire you?

Hey everyone! My name is Natalie and I'm an editor of the Atlassian Blog and I've got a question for you: What's your favorite quote about teamwork?  We've compiled a list here, along with...

160 views 16 7
Join discussion

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you