It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

How do I list all available disks in Linux?

Sam Caldwell Atlassian Team Mar 14, 2015

Problem:

When building automation, sometimes we need to iterate through all of the available disks on the system.  What's a quick and effective way to do this?

2 answers

1 accepted

man df

df -kh

Sam Caldwell Atlassian Team Mar 16, 2015

Matt, df is a great command. I use it a lot. But I'm not sure it works for this use case. For example, when I run it against one server, I get {code} scaldwell@reallysecretnode:~$ df -kh Filesystem Size Used Avail Use% Mounted on /dev/mapper/hidden--vg-root 00G 0.1G 14G 24% / udev 0.0G 0.0K 0.9G 1% /dev tmpfs 0.0G 000K 0.6G 1% /run none 0.0M 0 0.0M 0% /run/lock none 0.0G 0 0.9G 0% /run/shm /dev/mapper/something 0.0G 00M 0.0G 0% /tmp /dev/mapper/obfuscated 00G 00M 00G 0% /opt/obfuscated /dev/mapper/data 00G 00G 00G 00% /opt/j2ee/hidden /dev/mapper/obfuscated 00G 00M 00G 0% /opt/secret /dev/xda1 00M 00M 00M 00% /obfuscated {code} What I need in my particular case is to be able to separate disks from partitions and such. This is why I chose lsblk instead of df. Is there a right answer? Not really. I guess it depends on the specific user case. In either case, Matt, you have raised a great alternative. The next question is timing and performance. If we ignore use-case specifics and assume that df or lsblk are equivalent, then we would probably want to know which command runs faster. I did a quick test between meetings and found that df takes 0.003s on average. Here's where Matt's suggestion wins it: lsblk takes 0.005s to finish on average. It all comes down to use-case first then performance.

0 votes
Sam Caldwell Atlassian Team Mar 14, 2015

I was working on this problem today for a project.  Here's what I came up with...

for disk in $(lsblk -l | grep disk | awk '{print $1}'); do 
	#
	# Do work on the disk here.
	# 	
done

 

For example, in the case of my current project, I want to mount all of these partitions to /mnt/glusterfs/$disk...

for disk in $(lsblk -l | grep disk | awk '{print $1}'); do 
	mkdir -p /mnt/glusterfs/$disk && \
	mount /dev/$disk /mnt/glusterfs/$disk && \
	echo "$disk mounted successfully."	
done

Now the above exists in many places on the Internet, but let's add a twist to separate the rock stars from the newbies...

Assume you want to operate only on the disks that are NOT mounted to the system (because when we are deploying systems, we almost always have a mounted file system).  How do we filter out these already-mounted and healthy file systems?

for i in $(lsblk -l | grep disk | awk '{print $1}');do 
	if [ $(lsof | egrep $i[0-9]+ | wc -l) -eq 0 ]
	then
		echo "$i is not mounted."
	else
		echo "$i is mounted."
	fi
done

Now that's a little bit more awesomeness.  We know that because the file system is mounted, then a DIR (directory) and one or more files will be open.  Our friend `lsof` always tells us about open files, so testing the output of this command with a simple regex and counting the number of lines (wc -l) in the result gives us an idea as to whether or not a disk is in use.  That means we can do something like this to a virtual machine deployment we are automating....

for i in $(lsblk -l | grep disk | awk '{print $1}');do 
	[ $(lsof | egrep $i[0-9]+ | wc -l) -eq 0 ] && {
		echo "y" | mkfs.ext4 /dev/$i
        mkdir -p /dev/glusterfs/$i
        mount dev/$i /dev/glusterfs/$i
	}
done

In the above example, we only care about disks that are not in use.  We identify these disks and then use `mkfs.ext4` to format the disk with the ext4 file system.  Since we want an automated job, we echo a "y" character and pipe this to `mkfs.ext4` to avoid being prompted for any confirmation.

Have fun!

Sam Caldwell Atlassian Team Mar 14, 2015

Disclaimer: As a fellow Atlassian SOPS Engineer (ahem, thanks Harold) pointed out just now, `lsof` is expensive. Using lsof -n will save you a considerable amount of effort. However, using the `lsof` command in a loop should be avoided in most use cases. In the above examples, the expense of `lsof` is mitigated by the fact that it is to be used in a machine deployment scenario 10s after boot.

Suggest an answer

Log in or Sign up to answer
Community showcase
Posted in Jira

We want to hear from you! How do you use Slack and Atlassian together?

Hi Everyone! My name is Mina and I am on Atlassian’s Ecosystems Marketing team. Our team is focused on our technology partnerships and marketplace apps. One of Atlassian’s partners is Slack, who ...

44 views 0 4
Join discussion

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you