Search the Community

This is a collection of free courses, books, projects, repositories, cheat sheets, and online compilers on Python to help you get started and gain experience.View the full article

Lucene query cheatsheet Basic Search Single Term:term Finds documents containing term. Phrase Search:"exact phrase" Finds documents containing the exact phrase. Boolean Operators AND:term1 AND term2 Both terms must be present. OR:term1 OR term2 At least one of the terms must be present. NOT:NOT term Documents must not contain term. Combination:(term1 AND term2) OR term3 Complex boolean logic can be applied by combining operators. Wildcard Searches Single Character Wildcard:te?t Matches text with one character replaced. Multiple Character Wildcard:test* Matches text with zero or more characters. Wildcard at Start:*test Not supported directly but can be used in certain contexts. Fuzzy Searches Fuzzy:term~ Matches terms that are similar to the specified term. Proximity Searches Proximity:"term1 term2"~N Matches terms that are within N words of each other. Range Searches Range:[start TO end] Finds documents with terms within the specified range. Exclusive Range:{start TO end} Excludes the exact start and end values. Regular Expressions Regex:/regex/ Matches terms by regular expression. Boosting Terms Boost:term^N Increases the relevance of a term by a factor of N. Field-Specific Searches Specific Field:fieldname:term Searches for the term within a specific field. Grouping Group Queries:(query1) AND (query2) Groups parts of queries for complex searches. How to search Apache HTTPD using Lucene These examples assume that the logs have been indexed in a Lucene-based system like Elasticsearch, and they demonstrate how to utilize various Lucene query features to filter and search log data effectively. Note that the specific fields used in these examples (ip, timestamp, response, request, etc.) should correspond to the fields defined in your Lucene schema for Apache HTTPD logs. // 1. Find logs for a specific IP address ip:"192.168.1.1" // 2. Search logs within a specific date range timestamp:[20230101 TO 20230131] // 3. Identify logs with 4xx client error response codes response:[400 TO 499] // 4. Locate logs for requests to a specific URL request:"GET /index.html HTTP/1.1" // 5. Filter logs by a specific user-agent string agent:"Mozilla/5.0 (Windows NT 10.0; Win64; x64)" // 6. Search for logs with a specific referrer referrer:"http://example.com/" // 7. Find all logs of GET requests request_method:GET // 8. Filter logs resulting in 5xx server errors response:[500 TO 599] // 9. Identify requests to a specific directory request:"/images/*" // 10. Locate requests taking longer than 2 seconds duration:>2000 // 11. Exclude logs from a specific IP address -ip:"192.168.1.1" // 12. Find requests for a specific file type (.jpg) request:"*.jpg" // 13. Identify logs from a specific day timestamp:20230115 // 14. Search logs with responses in a byte range bytes:[1000 TO 5000] // 15. Filter logs by HTTP method and response code request_method:POST AND response:200 // 16. Search for failed login attempts (custom log message) message:"Failed login attempt" // 17. Find logs from a range of IP addresses ip:[192.168.1.1 TO 192.168.1.100] // 18. Identify logs with a 200 OK response response:200 // 19. Search for logs with specific query parameters request:"*?user=john&*" // 20. Locate logs with a 404 Not Found response response:404 The post Apache Lucene Query Example appeared first on DevOpsSchool.com. View the full article

KDnuggets' latest original cheat sheet covers Jupyter Notebook magic methods. Check it out now and become a notebook magician.View the full article

Learn everything you need to know about Copilot’s Bing AI chatbot (formerly Bing Chat), its features, capabilities, limitations and more.View the full article

As the title says, this article will list my cheat sheet for ModelMapper. It will not provide any deep-dive tutorials or fancy descriptions, just some use cases. Models Used for This Article Any time you see User, UserDTO, LocationDTO in the code, refer to this section. View the full article

When will Apple's Vision Pro AR headset be released? How much will the new Apple VR headset cost? Find out what you need to know with our cheat sheet.View the full article

KDnuggets has brought together all of its in-house cheat sheets from 2023 in this single, convenient location. Have a look to make sure you didn't miss out on anything over the year.View the full article

Everything you need to know to get started with Bard, Google’s experimental conversational AI chatbot.View the full article

How much better is GPT-4 compared to previous models? Learn about cost and capabilities.View the full article

Graph databases are quickly becoming a core part of the analytics toolset for enterprise IT organizations. If you know SQL, you can easily learn Cypher and open up a huge opportunity for data analysis.View the full article

Microsoft PowerToys is a free download that can manage application windows, rename files in bulk, preview file contents and more.View the full article

Zero-day exploits use unknown vulnerabilities to infiltrate PCs, networks, mobile phones and IoT devices. For unprepared security teams, these exploits bring financial consequences and long-term risks.View the full article

Get up and running with ChatGPT with this comprehensive cheat sheet. Learn everything from how to sign up for free to enterprise use cases, and start using ChatGPT quickly and effectively.View the full article

This is a comprehensive guide for Microsoft 365 Copilot. Learn more about the release date, pricing, benefits and more.View the full article

The post Cheat – The Ultimate Linux Commands Cheat Sheet for Beginners and Administrators first appeared on Tecmint: Linux Howtos, Tutorials & Guides .What you do when you are not sure of the command you are running especially in case of complex Linux commands which use a lot The post Cheat – The Ultimate Linux Commands Cheat Sheet for Beginners and Administrators first appeared on Tecmint: Linux Howtos, Tutorials & Guides.View the full article

Docker is a well-liked platform for project development and deployment. The docker containerization concept has made it one of the most popular tools for building projects and apps. The major components around which Docker plays are Containers, Docker images, Networks, and Docker registry. With the help of Docker containers, it is simple to install each and every project requirement into a container and fix deployment problems like missing libraries and dependencies, incorrectly specified paths, and many more. While using Docker, the user must understand Docker elements and its basic commands to manage and use Docker components. This post will provide a basic command line cheat sheet for Docker. Docker Cheat Sheet Docker CLI is one of the most effective ways to use and manage Docker components to build, ship, and develop projects and software. To get started with the Docker platform, we have provided some basic commands to manage the essential components of Docker. Docker Image Docker images are the base point of Docker that are used to containerize applications. The Docker image can be created using a simple instruction file, “Dockerfile”. To manage and build the Docker image, we have listed basic commands: Command Name Syntax Description Build Image docker build -t <img-name> -f Dockerfile . The “build” command creates or generates the Image from Dockerfile. Run Image docker run -d <img-name> Creates a new container and runs command or Docker image within a container. Run Image on Port docker run -d -p 8080:8080 <img-name> To run a container or image as a container on a specific port, the “-p” option is used along with the “run” command. List Image docker images -a List all Docker images. Tag Image docker tag <img-name> <new-img-name>:<tag> Tag the Docker image to uniquely identify the image version. Remove Image docker rmi -f <img-name> This command is used to remove images forcefully. Image History docker history <img-name> This command shows the detailed history of the Docker image. View Supported Options for Image Build docker build –help This command shows the supported options for the Docker build. Docker Containers Docker containers are another significant part of the Docker environment. These are used to manage projects and their dependencies. Containers encapsulate all project essentials and source code to build, deploy, and share projects. To manage and use containers in Docker, go through the provided table: Command Name Syntax Description Create Container docker create –name <container-name> -p 5000:5000 <docker-img> This command is used to create the Docker container. List Container docker ps -a This command is used to list all containers. Start Container docker start <container-name> This command starts the container. However, users can also use container id with the “start” command. Stop Containers docker stop <container-name> The provided command will stop the executing container. Remove Container docker rm <container-name> To remove the container, the “docker rm” command is used. Restart Container docker restart <container-name> This command will restart the stopped container. Kill Container docker kill <container-name> The “kill” command kills the running containers only. Kill All Running Containers docker kill $ (docker ps -q) This command will kill or eliminate all running containers. Attach Container docker attach <container-name> Connect a running container’s local input, output, and error streams. Exposed Port docker port <container-name> Show the mapping of ports within the container. Docker Registry Docker registry is the cloud storage and system to manage, publish and store Docker images. For this purpose, Docker has provided us with the official registry “Docker Hub”. To play with the Docker registry from the command line, check out the provided commands: Command Name Syntax Description Login docker login This command is used to log in to Docker Hub. Users can also use the “-u” option to provide the user name in the command. Logout docker logout This command logs out the user from the Docker registry. Search Image docker search <img-name> This command is used to search images from the Docker registry. Push Image docker push <img-name> This command is used to push the Docker image from the local registry to the remote registry, either in the private or official Docker registry. Pull Image docker pull <img-name> This command is utilized for pulling or downloading images from the Docker registry. Docker Volume The file system that is linked to the Docker container is referred to as the Docker Volume. It is used to preserve the data produced by the Docker container. To manage the Docker volume, look at Docker volume basic commands: Command Name Syntax Description Create Volume docker volume create <volume-name> This command creates the new volume. List Volume docker volume ls List all Docker volumes. Remove Volume docker volume rm -f <volume-name> This command is used to remove volume forcefully. Docker Network Docker network provides the network on which users can execute services or projects. Docker networking enables us to use as many networks as we like. To get started with the Docker network, look at the provided table: Command Name Syntax Description Create Network docker network create <option> <network-name> This command creates a new network. List Network docker network ls List all available networks. Inspect Network docker network inspect <network> This command shows detailed information on one or more networks. Connect Network docker network connect network container This command is utilized to connect the network with the container Remove Network docker network rm <network> This command removes the network. Docker Clean While working on the Docker platform for project development, users are occasionally required to clean the Docker system or maybe clean some components of Docker. For this purpose, go through the mentioned commands in the table: Command Name Syntax Description Docker Prune Volume docker volume prune This command prunes or removes all unused volume in Docker. Docker Prune Image docker image prune -a This command removes all dangling or unused Docker images. Docker Prune Container docker container prune -a The provided command removed all dangling, unused, and stopped containers. Docker Prune System docker system prune The specified command completely cleans the Docker by removing all unused, dangling Docker images, networks, and containers. To remove volume along with other components, the “–volume” option will be used. Remove All Containers docker rm $(docker ps -aq) This command will remove all stopped Docker containers. Remove All Images docker rmi -f $(docker images -aq) The provided command will remove all Docker images forcefully. Here we go! We have provided the basic command line cheat sheet for the Docker platform. Conclusion Docker CLI is one of the most effective ways to use and manage Docker components to build, ship, and develop projects and software. To play with Docker, the user must have basic knowledge of Docker commands to build, create, run, and remove the Docker images, containers, volume, and networks. This blog has provided the basic command line cheat sheet for Docker. View the full article

https://kubesimplify.com/get-good-at-git

Helm is a Kubernetes package manager for deploying helm charts (collections of pre-configured Kubernetes application resources). It features all the necessary commands for the simpler management of apps in a Kubernetes cluster. This article covers all important Helm operations and provides examples to help you understand its syntax and features. https://faun.pub/helm-command-cheat-sheet-by-m-sharma-488706ecf131

Hello everyone, here is a list of all the linux commands I can think of and can be helpful for your usage. Input/output echo It is used to print string values in the terminal. Examples: echo "Hello World" #Print with a newline echo -n "Hello World" #Print without a newline echo -e "Hello\tWorld" #Print with tab space printf It is used to print formatted string values in the terminal. Examples: printf "Hello World" #Print without a newline printf "Hello World\n" #Print with a newline printf "The price of the book is %0.2f\n" 56.856 #Print the formatted data read It is used to take input from the user. Examples: read variable #Take single input read variable1, variable2, variable3 #Take multiple inputs read - p "enter your name: " variable #Take input with prompt read - sp "enter password: " variable #Take secret input read - t 5 - p "Enter your pin : " pin_number #Take input with time limit read - n 11 mobile #Take input of maximum length pipe It is used to take the output of one process and pass it as the input for another process. The symbol, ‘|’ is for the pipe. Example: # The output of`echo` command will be sent as the input of the`wc` command echo "Hello World" | wc –w Redirection The redirection is used in Linux to change the standard input/output of the device after executing the command. The ‘<’ symbol is used for input redirection and the ‘>’ symbol is used for output redirection. Examples: ls - l > folderList.txt #Redirect output ehco "test" 2 > error.txt #Redirect error into the file ehco "test" 2 >& 1 error.txt #Redirect error into the file and the terminal cat > myfile.txt < folderList.txt #Redirect input data from one file to another file System Information date It is used to print the current date and time of the system in different formats. Examples: date #Print current date and time without formatting date +%m-%d-%Y #Print the current date date +%H:%M:%S #Print the current time date +"%A, %B %d,%Y" #Print the current date with weekday and month name hostname It is used to check or modify the hostname, domain, and IP address. Examples: hostname #Display the hostname hostname -d #Display the domain name hostname -I #Display the IP address hostname -b new_hostname #Set the new hostname uname It is used to print different information about the hardware and the operating system. Examples: uname #Print the kernel name uname –n #Print the hostname uname –m #Print the hardware name uname -o #Print the operating system name uname –a #Print all information related to hardware and OS vmstate It is used to get information about the memory, disk, processes, block IO, paging, and CPU scheduling. Examples: vmstat –a #Display the active and inactive memory of the system vmstat –f #Display the number of forks since boot vmstat -s #Display event counter and memory statistics vmstat -d #Display disk statistics iostate It is used to monitor the statistics of the system input/output of devices and partitions. The sysstat package requires to install before using this command. Examples: Iostat #Print all statistics and report Iostat -c #Print the statistic information of CPU Iostat -d #Print the device report Iostat -x #Print the statistics information in details Iostat -k # Print the statistic in KB or MB Iostat -p #Print the statistics information of block devices Iostat -N #Print the statistics information of lvm2 free It is used to print the detailed report of the system’s memory usage. Examples: free #Print the output in the default format free -h #Print the output in human-readable format free -b #Print the output in bytes free -k #Print the output in KB free -g #Print the output in GB whoami It is used to print the name of the currently logged-in user. Example: whoami #Print the logged-in user's name uptime It is used to print the information about the total running time of the system. Examples: uptime #Print the uptime in the default format uptime -p #Print the uptime in pretty format uptime –s #Print the time since the system is up shutdown It is used to power off or shutdown or restart the system from the terminal. Examples: shutdown #Shutdown normally shutdown –h 0 #Shutdown immediately shutdown –r #Shutdown and restart shutdown –r now #Shutdown and restart immediately shutdown –c #Shutdown can be canceled reboot It is used to reboot the system from the terminal with root privilege. Example: sudo reboot #Reboot the system User Management adduser It is used to add a new user account in the Linux system. The root privilege requires to execute this command. Examples: #Create a new user information in default location sudo adduser username #Create a new user information in the specified location sudo adduser username --home /home/temp/ deluser It is used to remove the Linux user account. The root privilege requires to execute this command. Examples: sudo deluser username #Delete user sudo deluser --remove-home username #Delete user with home directory sudo deluser –force #Delete user forcefully sudo deluser --backup-to /backupUser username #Delete user keeping backup files usermod It is used to modify the information of any existing Linux user. It requires root privilege to execute. Examples: sudo usermod --login newUsername oldUsername #Change the username sudo usermod --uid newID username #Change the ID of an existing user sudo usermod --gid groupname username #Change the group name of an existing user groupadd It is used to create a new group. It requires root privilege to execute this command. Examples: #Create a new group sudo groupadd groupname #Create a group forcefully if it already exists sudo groupadd –f groupname #Create a new group with a particular group id sudo groupadd -g 6723 groupname #Create a new group by defining the range of group id sudo groupadd -K GID_MIN=4000 -K GID_MAX=5000 groupname groupdel It is used to delete an existing group. It requires root privilege to execute this command. Examples: sudo groupdel groupname #Delete the group name sudo groupdel –f groupname #Delete the group forcefully groupmod It is used to modify any existing group. It requires root privilege to execute this command.Examples: sudo groupmod -n newGroupname Groupname #Change the group name sudo groupmod -g 676 Groupname #Change the group id last It is used to print the information about the last login session of the Linux users. Examples: last #Print all last login session information last username #Print the last login session information of a user last -10 #Print the last 10 login session last –p 2022-01-01 #Print the last login session information of a specific date id IT is used to search the username, group name, and the ID of the current user or another user of the server. Examples: id #Print the ID and group information related to the current login user’s id username #Print the ID and group information related to the particular user id -u username #Print the user ID of the particular user id -g username #Print the group ID of the particular user groups It is used to print the primary and supplementary group names for each username or the current process if no username is given. Examples: Groups #Print the group information of the current login user groups username #Print the group information of the specific user groups username1 username2 #Print the group information of the multiple users passwd It is used to change the user’s password. The root user can change the password of all users and other users can change their password only. Examples: passwd #Change the password of the current login user sudo passwd username #Change the password of any user with root privilege sudo It is used to perform the restricted tasks with the root privileges without logging as root user or by logging as the root user. Examples: sudo apt-get update #Update the current system sudo -i #Login as the root user sudo bash #Run bash as the root user sudu !! #Execute the last command with the root privilege su It is used to switch from the current user to another user. Example: su username #Switch to another user without changing the home directory su -l username #Switch to another user by changing the current directory su -p username #Switch to another user by preserving the environment variables Packages apt It is used to install, update, delete and manage different packages in Linux operating system. It requires root privileges. Examples: sudo apt update #Update the installed packages sudo apt upgrade #Upgrade the installed packages sudo apt upgrade packagename #Upgrade specific package apt-get It works like the `apt` command but some commands are used differently by `apt-get` command when it is used in replacement of the `apt` command. Examples: sudo apt-get update #Update the packages sudo apt-get dist-upgrade #Upgrade the packages with full dependencies dpkg It is used to install, remove, manage and query Debian packages and their dependencies. It requires root privilege to use. Examples: #Install the Debian package sudo dpkg –i debian_package_name #Print installed packages related to nano editor sudo dpkg -l '*nano*' #Remove vim editor sudo dpkg -r vim Disk Usage du It is used to get disk usage information. Examples: du #Print the disk usage information in the default format du -h #Print the disk usage information in human-readable format du -c #Print the disk usage information with total measurement value. du -s #Print the summary of the disk usage information du -a #Print disk usage information with the sizes of files and folders df It is used to print the used available disk space information of the file system. Examples: df #Print the used and available disk space in the default format df -h #Print the used and available disk space in human-readable format df -T #Print the used and available disk space with file system type df --total #Print the used and available disk space with total values df -a #Print the used and available disk space with other information fdisk It is used to create or modify the partition tables of the hard disk. Examples: fdisk -l #Print partition table of all devices fdisk /dev/sdb #Run fdisk for the specific device fdisk -l /dev/sda #Print the partition table list of a device mount It is used to attach the file system and removable devices at a particular mount point. Examples: sudo mount #Print all currently attached file system sudo mount -t ext3 #Print the list of all ext3 file system sudo mount /dev/sda1 /mnt/media #Mount a file system and a directory unmount It is used to detach the mounted file system and the device. Examples: sudo umount ext3 #Unmount ext3 file system sudo umount /dev/sdb1 #Unmount sdb1 device Process Management ps It is used to get information related to the processes of the system. Examples: ps #Print the process list of the current shell ps -e #Print the list of all running process ps -a #Print the list of the process that is not associated with the terminal ps -T #Print the list of the process that is associated with the terminal pgrep It is used to search the process ID of the running program based on the specific condition. Examples: pgrep bash #Print the process ID pgrep ssh -l #Print the process name with ID pgrep -u root #Print the process list of a particular user kill It is used to send a signal to a process defined by a PID. The default signal is SIGTERM which terminates the process. Examples: kill –l #Print the available kill signals kill 5645 #Terminate the process based on ID kill -1 9078 #Reload the process based on the ID kill –n 15 7845 8756 #Terminate multiple processes killall It is used to kill any running process forcefully based on the given process name. Examples: killall processname #Kill the process based on name killall -i processname #Ask permission before killing the process killall –u username #Kill process based on the username pkill It is used to send a signal to a process of the running program based on the full or partial process name, username, or other attributes. Examples: pkill firfox #Terminate a particular process pkill -i gedit #Terminate a process in a case-insensitive manner fg It is used to move the background job of the current shell into the foreground. Example: fg %1 #Refer to the specific job number fg %+ #Refer to the current job fg %- #Refer to the previous job bg It is used to restart a stopped background process. Example: bg %1 #Refer to the specific job number bg %+ #Refer to the current job bg %- #Refer to the previous job Scripting Tools sed It is used to search, find-replace, and insert or delete purposes. Examples: #Replace the word, 'World' with the word 'Everyone' echo 'Hello World' | sed 's/World/Everyone/' #Replace the word, 'book' with the word 'e-book' sed 's/book/e-book/g' filename.txt #Delete the 5th line of the file sed '5d' filename.txt awk It is a scripting language that is used for manipulating data and generating a report by pattern scanning and processing. Example: awk '{print}' filename.txt #Print the content of the file awk '/Linux/ {print}' filename.txt #Print those lines of the file that contains ‘Linux’ awk '{print $1,$4}' filename.txt #Print the first and fourth columns of the file sort It is used to sort the content of a file. Examples: sort filename.txt #Sort the string data in ascending order sort –r filename.txt #Sort the string data in descending order sort -n file1.txt #Sort the numeric data in ascending order sort -o output.txt filename.txt #Redirect the sorted output of filename.txt to output.txt wc It is used to count the number of characters, words, and lines of a file or the standard input. Examples: echo "Hello" | wc –c #Count the total characters of the string wc –w filename.txt #Count the total words of a file wc –l filename.txt #Count the total lines of a file uniq It is used to filter the duplicate lines of a file. Examples: uniq filename.txt #Print the file after filtering the duplicate lines uniq –c filename.txt #Print the file after filtering the duplicate lines with number uniq –d filename.txt #Print the duplicate lines of the file only. uniq –u filename.txt #Print the unique lines of the file only tr It is used to convert the string or delete characters from the string. Examples: echo Linux | tr [:lower:] [:upper:] #Change all small letters to capital letters of the string tr A-Z a-z < filename.txt #Change all capital letters to small letters of a file echo "fool" | tr -c 'ool\n' 'b' #Change those characters by ‘b’ that don’t match with ‘ool\n’ File commands touch It is used to create an empty file and change the access or modification timestamp value of the existing file. Examples: touch filename #Create a blank file touch file1, file2, file3 #Create multiple blank files touch -a filename.txt #Modify the access time of an existing file touch -m filename.txt #Modify the modification time of an existing file cat It is used to create, read, and concatenate files. Examples: cat > filename #Create a new file cat filename #Print an existing file cat filename1, filename2, filename3 #Print multiple files cat -n filename #Print an existing file with the line number cp It is used to copy file or group of files or directories. Examples: cp file1.txt file2.txt #Copy the file1.txt to file2.txt cp file1.txt file2.txt dirname #Copy multiple files into a directory cp -i file1.txt file2.txt #Ask before overwriting the existing file cp -f file1.txt file2.txt #Copy file forcefully mv It is used to move files and directories from one location to another location. Examples: mv filename1 filename2 #Move the file mv dircetory1 directory2 #Move the directory mv -i filename directory #Ask for overwriting before moving rm It is used to remove files and directories. Examples: rm filename #Remove a single file rm file1, file2, file3 #Remove multiple files rm -i filename #Ask before deleting rm -r directory #Remove all files and folders of the directory recursively rm -d directory #Remove empty directory ln It is used to create the symbolic link. Examples: ln -s filename #Create a link to a file ln -s filename linkname #Create a link with the linkname ln -sf filename linkname #Create a link forcefully if exists less It is used to read the large file or command output that displays the content of a file or a command output part by part to make the reading task faster. Examples: Less filename #Print the content of a file ps aux | less #Print the command output ps aux | less –N #Print the command output with line number ps aux | less -p "root" #Highlight the word, root in the output more It works similar to `less` command and it has many other features to scroll the content of the large file or command output. Examples: more file.txt #Display the content of a file more -d file..txt #Help user to navigate the content more -p file..txt #Display the page after clearing the previous content more -c file..txt #Display all pages in the same area dmesg | more #Display the content of command output cut It is used to cut the portion of each line from a file and print the output in the terminal. Examples: cut -b 3 file.txt #Print the 3rd character from each line of the file cut -c 2,5 file.txt #Print the 2nd and 5th characters from each line of the file cut -c -3 file.txt #Print the first 3 characters from each line of the file head It is used to read the file from the beginning. Examples: head -n 5 file.txt #Print the first 5 lines of the file head -n -5 file.txt #Print the lines after omitting the last 5 lines head -c 50 file.txt #Print the first 50 characters of the file head -q file1.txt file2.txt #Print the content of multiple files tail It is used to read the file from the end. Examples: tail -n 4 file.txt #Print the last 4 lines of the file tail -n -7 file.txt #Print the lines after omitting the first 7 lines tail -c 50 file.txt #Print the last 50 characters of the file tail -n 5 file1.txt file2.txt #Print the last 5 lines of two files File Permissions chmod It is used to set the permission bits for the files and folders. Examples: #Owner can read & write, the group users can read and others can execute chmod 642 filename #Owner can read, write and execute, the group users can execute and others can read only chmod u=rwx,g=x,o=r filename #Owner can read and write, the group users can write and others can do nothing to the files and folders #of the directory chmod -R 620 dirname chown It is used to change the ownership of a file for the owner or the group users. It requires root privileges to execute. Examples: sudo chown -c username filename #Change the ownership of a file sudo chown :groupname filename #Change the ownership of a group #Change the ownership by mentioning the usernames sudo chown --from=fahmida root linuxhint File Compression and Decompression gzip It is used to compress the file by keeping the original file mode, timestamp and ownership. Examples: gzip filename #Compress single file gzip file1 file2 file3 #Compress multiple files gzip -v filename #Display the percentage of compression gzip -r directory #Compress all files of a directory tar It is used to create compressed achieve files of large files and folders that can be transferred from one location to another.Examples: tar -cvf filename.tar /home/fahmida/ #Create a compress file of .tar extension tar -cvzf filename.tgz /home/fahmida/ #Create a compress file of .tgz extension tar -cvfj filename.bz2 /home/fahmida/ #Create a compress file of .bz2 extension zip It is used to compress the file in .zip format that is supported by many operating systems. Examples: zip file.zip file.txt #Compress the file with .zip extension zip -d file.zip file.txt #Remove the file from the zip archive zip -u file.zip file.txt #Update the file in the zip archive unzip It is used to unzip the files and folders from the zip achieve. Examples: unzip file.zip #Decompress file in the current location from the zip archive sudo unzip file.zip -d /temp #Decompress file in the particular folder from the zip archive Directory operations pwd It is used to know the path of the current working directory. Examples: pwd #Display the current working directory pwd –L #Display the logical working directory pwd –P #Display the physical working directory mkdir It is used to create a new directory. Examples: mkdir dirname #Create a single directory mkdir dir1, dir2, dir3 #Create multiple directory mkdir –m777 dirname #Create a directory with permission rmdir It is used to remove any directory. Examples: rmdir dirname #Remove single directory rmdir dir1, dir2, dir3 #Remove multiple directory #Remove the child directory first and remove the parent directory rmdir -p parent_dir/child_dir ls It is used to print the list of files and folders of the current path or a particular path. Examples: ls #Display the files and folders in the default format ls -a #Display files and folders with hidden files ls -l #Display the files and folders with the detailed information ls -h #Display the files and folders in human-readable format ls -r #Display the files and folders in reverse order ls -F #Display folders with ‘/’ cd It is used to change the directory. Examples: cd dirname #Change to a directory cd dir1/dir2 #Change to a directory that is inside another directory cd / #Change to the root directory cd #Change to the home directory Searching Commands grep It is used to search the particular string in a file. Examples: grep 'test' file.txt #Print the lines that match the word, 'test' grep -i ‘test’ file.txt #Print the lines that match the word, 'test' case-insensitively grep ‘^test’ file.txt #Print the lines that match the word, 'test' at the beginning of the line grep ‘test$’ file.txt #Print the lines that match the word, 'test' at the end of the line locate It is used to search a file by name. Examples: locate filename #Search a particular file locate "*.txt" -n 15 #Search text files with the limit locate -c [.sh]* #Count the number of matches locate -i filename #Search a file in a case-insensitive manner find It is used to search and locate the files and folders based on the condition. Examples: find . -name filename.txt #Search a file in the current directory find . -iname filename.txt #Search a file case-insensitively find /temp -name filename.txt #Search the file in the particular folder find / -type d -name dirname #Search the directory in ‘/’ folder whereis It is used to search the location of the source, binary, and the manual page files for a command. Examples: whereis -l #Find the directories where the `whereis` command will search whereis bash #Find the location of the `bash` whereis man uptime #Find the location of `man` and `uptime` Network Information ping It is used to test a host is reachable or not in the network. Examples: ping www.google.com #Test using the URL address ping 142.250.199.36 #Test using IP address ping -c 5 www.google.com #Test will stop after sending 5 packets ping -i 3 www.google.com #Test using 3 seconds interval ifconfig It is used to configure, manage, and query network interface parameters by using a command-line interface or system configuration script. You have to install the net-tools package to use this command. Examples: ifconfig #Display all active interfaces ifconfig -a #Display all active or inactive interfaces ifconfig eth0 #Display information about the specific interface ifconfig eth0 up #Activate the particular interface ifconfig eth0 down #Deactivate the particular interface ifconfig eth0 172.16.25.125 #Set the IP address for the particular interface dig It is used to search the DNS name servers for DNS lookup. Examples: dig example.com #Find the IP address of the hostname dig 93.184.216.34 #Find the hostname of the IP address dig example.com MX #Find the MX of a hostname netstat It is used to get information about the network activities and display which ports are open or have established connections. It is a useful tool to debug network problems. Examples: netstat –a #Print all ports and connections netstat –at #Print all TCP ports netstat –au #Print all UDP ports netstat -l #Print all listening ports of all protocols ip It is used to bring the interface up or down, assign and remove the address, etc. Examples: ip addr show #Display information of all IP addresses ip addr show dev eth0 #Display information about a single interface sudo ip address add 192.168.10.50/24 dev eth0 #Assign IP to an interface Bash Shortcut Keys alt+ctrl+t It is used to open a new terminal window. ctrl+c It is used to stop the running command. ctrl+a It is used to go to the start of the line. ctrl+e It is used to go to the end of the line. ctrl+u It is used to cut the line before the cursor to the clipboard. ctrl+k It is used to cut the line after the cursor to the clipboard. ctrl+r It is used to search history. ctrl+z It is used to suspend the current foreground process. !! It is used to repeat the last command. !$ It is used to read the last argument of the previous command. !* It is used to read all arguments of the previous command. Bash Script Basics Variables The variable is used to store and read data. Example: website="linuxhint.com" #Assign a string value to a variable echo $website #Print the value of the variable echo "The website name is $website" #Print the variable with another string Length The length of a string can be calculated in bash in different ways. The simple way to count the length is shown below. Example: var="Hello World" #Assign String value echo ${#var} #Print the length of the sting Substring Sometimes it requires cutting the portion of a string that is called substring. Example: var="Hello World" #Assign String value echo "${var:0:5}" #Print the substring, ‘Hello’ echo "${var:6:5}" #Print the substring, ‘World’ Uppercase The string can be converted into uppercase by using `tr` command and ‘^^’ operator. Example: var="Hello" echo $var | tr [:lower:] [:upper:] #Convert to uppercase using `tr` echo ${var^^} #Convert to uppercase using '^^' Lowercase The string can be converted into lowercase by using `tr` command and ‘,,’ operator. Example: var="Hello" echo $var | tr [A-Z] [a-z] #Convert to lowercase using `tr` echo ${var,,} #Convert to lowercase using '^^' If Statement If statement is used to define the conditional statements. Example: str="file.txt" if [[ -f $str ]]; then #Check the file exists or not echo "$str is a file." elif [[ -d $str ]]; then #Check the directory exist or not echo "$str is a directory." else echo "$str is not a fir or directory." fi Case statement The case statement can be used as an alternative to if statement. Example: id=45 case $id in 67) echo "Group-1";; #Match with an ID 45 | 89) echo "Group-2";; #Match with any of two IDs 92 | 12 | 54) echo "Group-3" ;; #Match with any of three IDs *) echo "You are not selected." ;; esac For loop It is used to iterate some statements multiple times or read the array values. Example: for i in 23 67 45 11 89 #Iterate the loop for 5 times do echo "The value of i = $i" #Print each iteration value done While loop It is used as the alternative of for loop. Example: n=10 #Initialize a number while [ $n -le 20 ] #Iterate the loop until the value reaches to 20 do echo "n=$n" #Print the current value of $n n=$(( $n + 5 )) #Increment the value of $n by 5 done Function It is used to define a block of code with a name that can be executed anytime by calling the function name. Example: function_name() { #Define the function echo "Hello World" } echo $(function_name) #Call the function eval It is used to execute arguments like a shell command. Example: c1=$(( 10 + 15 )) #Define the argument c2="echo " #Define the second argument eval $c1 $c2 #Execute the arguments set It is used to control certain flags and features in Bash to set the behavior of the scripts. Examples: set –C #Disable the feature of overwriting the file set –e #Stop script immediately when an error occurs unset Ii is used to delete the variables during the program execution. It can be used to delete both functions and shell variables. Example: var="Hello World" #Assign String value echo $var #Print the assigned variable unset $var #Unset the variable echo $var #Print the variable after unset View the full article

Hello everyone, here is a list of all the linux commands I can think of and can be helpful for your usage. Input/output echo It is used to print string values in the terminal. Examples: echo "Hello World" #Print with a newline echo -n "Hello World" #Print without a newline echo -e "Hello\tWorld" #Print with tab space printf It is used to print formatted string values in the terminal. Examples: printf "Hello World" #Print without a newline printf "Hello World\n" #Print with a newline printf "The price of the book is %0.2f\n" 56.856 #Print the formatted data read It is used to take input from the user. Examples: read variable #Take single input read variable1, variable2, variable3 #Take multiple inputs read - p "enter your name: " variable #Take input with prompt read - sp "enter password: " variable #Take secret input read - t 5 - p "Enter your pin : " pin_number #Take input with time limit read - n 11 mobile #Take input of maximum length pipe It is used to take the output of one process and pass it as the input for another process. The symbol, ‘|’ is for the pipe. Example: # The output of`echo` command will be sent as the input of the`wc` command echo "Hello World" | wc –w Redirection The redirection is used in Linux to change the standard input/output of the device after executing the command. The ‘<’ symbol is used for input redirection and the ‘>’ symbol is used for output redirection. Examples: ls - l > folderList.txt #Redirect output ehco "test" 2 > error.txt #Redirect error into the file ehco "test" 2 >& 1 error.txt #Redirect error into the file and the terminal cat > myfile.txt < folderList.txt #Redirect input data from one file to another file System Information date It is used to print the current date and time of the system in different formats. Examples: date #Print current date and time without formatting date +%m-%d-%Y #Print the current date date +%H:%M:%S #Print the current time date +"%A, %B %d,%Y" #Print the current date with weekday and month name hostname It is used to check or modify the hostname, domain, and IP address. Examples: hostname #Display the hostname hostname -d #Display the domain name hostname -I #Display the IP address hostname -b new_hostname #Set the new hostname uname It is used to print different information about the hardware and the operating system. Examples: uname #Print the kernel name uname –n #Print the hostname uname –m #Print the hardware name uname -o #Print the operating system name uname –a #Print all information related to hardware and OS vmstate It is used to get information about the memory, disk, processes, block IO, paging, and CPU scheduling. Examples: vmstat –a #Display the active and inactive memory of the system vmstat –f #Display the number of forks since boot vmstat -s #Display event counter and memory statistics vmstat -d #Display disk statistics iostate It is used to monitor the statistics of the system input/output of devices and partitions. The sysstat package requires to install before using this command. Examples: Iostat #Print all statistics and report Iostat -c #Print the statistic information of CPU Iostat -d #Print the device report Iostat -x #Print the statistics information in details Iostat -k # Print the statistic in KB or MB Iostat -p #Print the statistics information of block devices Iostat -N #Print the statistics information of lvm2 free It is used to print the detailed report of the system’s memory usage. Examples: free #Print the output in the default format free -h #Print the output in human-readable format free -b #Print the output in bytes free -k #Print the output in KB free -g #Print the output in GB whoami It is used to print the name of the currently logged-in user. Example: whoami #Print the logged-in user's name uptime It is used to print the information about the total running time of the system. Examples: uptime #Print the uptime in the default format uptime -p #Print the uptime in pretty format uptime –s #Print the time since the system is up shutdown It is used to power off or shutdown or restart the system from the terminal. Examples: shutdown #Shutdown normally shutdown –h 0 #Shutdown immediately shutdown –r #Shutdown and restart shutdown –r now #Shutdown and restart immediately shutdown –c #Shutdown can be canceled reboot It is used to reboot the system from the terminal with root privilege. Example: sudo reboot #Reboot the system User Management adduser It is used to add a new user account in the Linux system. The root privilege requires to execute this command. Examples: #Create a new user information in default location sudo adduser username #Create a new user information in the specified location sudo adduser username --home /home/temp/ deluser It is used to remove the Linux user account. The root privilege requires to execute this command. Examples: sudo deluser username #Delete user sudo deluser --remove-home username #Delete user with home directory sudo deluser –force #Delete user forcefully sudo deluser --backup-to /backupUser username #Delete user keeping backup files usermod It is used to modify the information of any existing Linux user. It requires root privilege to execute. Examples: sudo usermod --login newUsername oldUsername #Change the username sudo usermod --uid newID username #Change the ID of an existing user sudo usermod --gid groupname username #Change the group name of an existing user groupadd It is used to create a new group. It requires root privilege to execute this command. Examples: #Create a new group sudo groupadd groupname #Create a group forcefully if it already exists sudo groupadd –f groupname #Create a new group with a particular group id sudo groupadd -g 6723 groupname #Create a new group by defining the range of group id sudo groupadd -K GID_MIN=4000 -K GID_MAX=5000 groupname groupdel It is used to delete an existing group. It requires root privilege to execute this command. Examples: sudo groupdel groupname #Delete the group name sudo groupdel –f groupname #Delete the group forcefully groupmod It is used to modify any existing group. It requires root privilege to execute this command.Examples: sudo groupmod -n newGroupname Groupname #Change the group name sudo groupmod -g 676 Groupname #Change the group id last It is used to print the information about the last login session of the Linux users. Examples: last #Print all last login session information last username #Print the last login session information of a user last -10 #Print the last 10 login session last –p 2022-01-01 #Print the last login session information of a specific date id IT is used to search the username, group name, and the ID of the current user or another user of the server. Examples: id #Print the ID and group information related to the current login user’s id username #Print the ID and group information related to the particular user id -u username #Print the user ID of the particular user id -g username #Print the group ID of the particular user groups It is used to print the primary and supplementary group names for each username or the current process if no username is given. Examples: Groups #Print the group information of the current login user groups username #Print the group information of the specific user groups username1 username2 #Print the group information of the multiple users passwd It is used to change the user’s password. The root user can change the password of all users and other users can change their password only. Examples: passwd #Change the password of the current login user sudo passwd username #Change the password of any user with root privilege sudo It is used to perform the restricted tasks with the root privileges without logging as root user or by logging as the root user. Examples: sudo apt-get update #Update the current system sudo -i #Login as the root user sudo bash #Run bash as the root user sudu !! #Execute the last command with the root privilege su It is used to switch from the current user to another user. Example: su username #Switch to another user without changing the home directory su -l username #Switch to another user by changing the current directory su -p username #Switch to another user by preserving the environment variables Packages apt It is used to install, update, delete and manage different packages in Linux operating system. It requires root privileges. Examples: sudo apt update #Update the installed packages sudo apt upgrade #Upgrade the installed packages sudo apt upgrade packagename #Upgrade specific package apt-get It works like the `apt` command but some commands are used differently by `apt-get` command when it is used in replacement of the `apt` command. Examples: sudo apt-get update #Update the packages sudo apt-get dist-upgrade #Upgrade the packages with full dependencies dpkg It is used to install, remove, manage and query Debian packages and their dependencies. It requires root privilege to use. Examples: #Install the Debian package sudo dpkg –i debian_package_name #Print installed packages related to nano editor sudo dpkg -l '*nano*' #Remove vim editor sudo dpkg -r vim Disk Usage du It is used to get disk usage information. Examples: du #Print the disk usage information in the default format du -h #Print the disk usage information in human-readable format du -c #Print the disk usage information with total measurement value. du -s #Print the summary of the disk usage information du -a #Print disk usage information with the sizes of files and folders df It is used to print the used available disk space information of the file system. Examples: df #Print the used and available disk space in the default format df -h #Print the used and available disk space in human-readable format df -T #Print the used and available disk space with file system type df --total #Print the used and available disk space with total values df -a #Print the used and available disk space with other information fdisk It is used to create or modify the partition tables of the hard disk. Examples: fdisk -l #Print partition table of all devices fdisk /dev/sdb #Run fdisk for the specific device fdisk -l /dev/sda #Print the partition table list of a device mount It is used to attach the file system and removable devices at a particular mount point. Examples: sudo mount #Print all currently attached file system sudo mount -t ext3 #Print the list of all ext3 file system sudo mount /dev/sda1 /mnt/media #Mount a file system and a directory unmount It is used to detach the mounted file system and the device. Examples: sudo umount ext3 #Unmount ext3 file system sudo umount /dev/sdb1 #Unmount sdb1 device Process Management ps It is used to get information related to the processes of the system. Examples: ps #Print the process list of the current shell ps -e #Print the list of all running process ps -a #Print the list of the process that is not associated with the terminal ps -T #Print the list of the process that is associated with the terminal pgrep It is used to search the process ID of the running program based on the specific condition. Examples: pgrep bash #Print the process ID pgrep ssh -l #Print the process name with ID pgrep -u root #Print the process list of a particular user kill It is used to send a signal to a process defined by a PID. The default signal is SIGTERM which terminates the process. Examples: kill –l #Print the available kill signals kill 5645 #Terminate the process based on ID kill -1 9078 #Reload the process based on the ID kill –n 15 7845 8756 #Terminate multiple processes killall It is used to kill any running process forcefully based on the given process name. Examples: killall processname #Kill the process based on name killall -i processname #Ask permission before killing the process killall –u username #Kill process based on the username pkill It is used to send a signal to a process of the running program based on the full or partial process name, username, or other attributes. Examples: pkill firfox #Terminate a particular process pkill -i gedit #Terminate a process in a case-insensitive manner fg It is used to move the background job of the current shell into the foreground. Example: fg %1 #Refer to the specific job number fg %+ #Refer to the current job fg %- #Refer to the previous job bg It is used to restart a stopped background process. Example: bg %1 #Refer to the specific job number bg %+ #Refer to the current job bg %- #Refer to the previous job Scripting Tools sed It is used to search, find-replace, and insert or delete purposes. Examples: #Replace the word, 'World' with the word 'Everyone' echo 'Hello World' | sed 's/World/Everyone/' #Replace the word, 'book' with the word 'e-book' sed 's/book/e-book/g' filename.txt #Delete the 5th line of the file sed '5d' filename.txt awk It is a scripting language that is used for manipulating data and generating a report by pattern scanning and processing. Example: awk '{print}' filename.txt #Print the content of the file awk '/Linux/ {print}' filename.txt #Print those lines of the file that contains ‘Linux’ awk '{print $1,$4}' filename.txt #Print the first and fourth columns of the file sort It is used to sort the content of a file. Examples: sort filename.txt #Sort the string data in ascending order sort –r filename.txt #Sort the string data in descending order sort -n file1.txt #Sort the numeric data in ascending order sort -o output.txt filename.txt #Redirect the sorted output of filename.txt to output.txt wc It is used to count the number of characters, words, and lines of a file or the standard input. Examples: echo "Hello" | wc –c #Count the total characters of the string wc –w filename.txt #Count the total words of a file wc –l filename.txt #Count the total lines of a file uniq It is used to filter the duplicate lines of a file. Examples: uniq filename.txt #Print the file after filtering the duplicate lines uniq –c filename.txt #Print the file after filtering the duplicate lines with number uniq –d filename.txt #Print the duplicate lines of the file only. uniq –u filename.txt #Print the unique lines of the file only tr It is used to convert the string or delete characters from the string. Examples: echo Linux | tr [:lower:] [:upper:] #Change all small letters to capital letters of the string tr A-Z a-z < filename.txt #Change all capital letters to small letters of a file echo "fool" | tr -c 'ool\n' 'b' #Change those characters by ‘b’ that don’t match with ‘ool\n’ File commands touch It is used to create an empty file and change the access or modification timestamp value of the existing file. Examples: touch filename #Create a blank file touch file1, file2, file3 #Create multiple blank files touch -a filename.txt #Modify the access time of an existing file touch -m filename.txt #Modify the modification time of an existing file cat It is used to create, read, and concatenate files. Examples: cat > filename #Create a new file cat filename #Print an existing file cat filename1, filename2, filename3 #Print multiple files cat -n filename #Print an existing file with the line number cp It is used to copy file or group of files or directories. Examples: cp file1.txt file2.txt #Copy the file1.txt to file2.txt cp file1.txt file2.txt dirname #Copy multiple files into a directory cp -i file1.txt file2.txt #Ask before overwriting the existing file cp -f file1.txt file2.txt #Copy file forcefully mv It is used to move files and directories from one location to another location. Examples: mv filename1 filename2 #Move the file mv dircetory1 directory2 #Move the directory mv -i filename directory #Ask for overwriting before moving rm It is used to remove files and directories. Examples: rm filename #Remove a single file rm file1, file2, file3 #Remove multiple files rm -i filename #Ask before deleting rm -r directory #Remove all files and folders of the directory recursively rm -d directory #Remove empty directory ln It is used to create the symbolic link. Examples: ln -s filename #Create a link to a file ln -s filename linkname #Create a link with the linkname ln -sf filename linkname #Create a link forcefully if exists less It is used to read the large file or command output that displays the content of a file or a command output part by part to make the reading task faster. Examples: Less filename #Print the content of a file ps aux | less #Print the command output ps aux | less –N #Print the command output with line number ps aux | less -p "root" #Highlight the word, root in the output more It works similar to `less` command and it has many other features to scroll the content of the large file or command output. Examples: more file.txt #Display the content of a file more -d file..txt #Help user to navigate the content more -p file..txt #Display the page after clearing the previous content more -c file..txt #Display all pages in the same area dmesg | more #Display the content of command output cut It is used to cut the portion of each line from a file and print the output in the terminal. Examples: cut -b 3 file.txt #Print the 3rd character from each line of the file cut -c 2,5 file.txt #Print the 2nd and 5th characters from each line of the file cut -c -3 file.txt #Print the first 3 characters from each line of the file head It is used to read the file from the beginning. Examples: head -n 5 file.txt #Print the first 5 lines of the file head -n -5 file.txt #Print the lines after omitting the last 5 lines head -c 50 file.txt #Print the first 50 characters of the file head -q file1.txt file2.txt #Print the content of multiple files tail It is used to read the file from the end. Examples: tail -n 4 file.txt #Print the last 4 lines of the file tail -n -7 file.txt #Print the lines after omitting the first 7 lines tail -c 50 file.txt #Print the last 50 characters of the file tail -n 5 file1.txt file2.txt #Print the last 5 lines of two files File Permissions chmod It is used to set the permission bits for the files and folders. Examples: #Owner can read & write, the group users can read and others can execute chmod 642 filename #Owner can read, write and execute, the group users can execute and others can read only chmod u=rwx,g=x,o=r filename #Owner can read and write, the group users can write and others can do nothing to the files and folders #of the directory chmod -R 620 dirname chown It is used to change the ownership of a file for the owner or the group users. It requires root privileges to execute. Examples: sudo chown -c username filename #Change the ownership of a file sudo chown :groupname filename #Change the ownership of a group #Change the ownership by mentioning the usernames sudo chown --from=fahmida root linuxhint File Compression and Decompression gzip It is used to compress the file by keeping the original file mode, timestamp and ownership. Examples: gzip filename #Compress single file gzip file1 file2 file3 #Compress multiple files gzip -v filename #Display the percentage of compression gzip -r directory #Compress all files of a directory tar It is used to create compressed achieve files of large files and folders that can be transferred from one location to another.Examples: tar -cvf filename.tar /home/fahmida/ #Create a compress file of .tar extension tar -cvzf filename.tgz /home/fahmida/ #Create a compress file of .tgz extension tar -cvfj filename.bz2 /home/fahmida/ #Create a compress file of .bz2 extension zip It is used to compress the file in .zip format that is supported by many operating systems. Examples: zip file.zip file.txt #Compress the file with .zip extension zip -d file.zip file.txt #Remove the file from the zip archive zip -u file.zip file.txt #Update the file in the zip archive unzip It is used to unzip the files and folders from the zip achieve. Examples: unzip file.zip #Decompress file in the current location from the zip archive sudo unzip file.zip -d /temp #Decompress file in the particular folder from the zip archive Directory operations pwd It is used to know the path of the current working directory. Examples: pwd #Display the current working directory pwd –L #Display the logical working directory pwd –P #Display the physical working directory mkdir It is used to create a new directory. Examples: mkdir dirname #Create a single directory mkdir dir1, dir2, dir3 #Create multiple directory mkdir –m777 dirname #Create a directory with permission rmdir It is used to remove any directory. Examples: rmdir dirname #Remove single directory rmdir dir1, dir2, dir3 #Remove multiple directory #Remove the child directory first and remove the parent directory rmdir -p parent_dir/child_dir ls It is used to print the list of files and folders of the current path or a particular path. Examples: ls #Display the files and folders in the default format ls -a #Display files and folders with hidden files ls -l #Display the files and folders with the detailed information ls -h #Display the files and folders in human-readable format ls -r #Display the files and folders in reverse order ls -F #Display folders with ‘/’ cd It is used to change the directory. Examples: cd dirname #Change to a directory cd dir1/dir2 #Change to a directory that is inside another directory cd / #Change to the root directory cd #Change to the home directory Searching Commands grep It is used to search the particular string in a file. Examples: grep 'test' file.txt #Print the lines that match the word, 'test' grep -i ‘test’ file.txt #Print the lines that match the word, 'test' case-insensitively grep ‘^test’ file.txt #Print the lines that match the word, 'test' at the beginning of the line grep ‘test$’ file.txt #Print the lines that match the word, 'test' at the end of the line locate It is used to search a file by name. Examples: locate filename #Search a particular file locate "*.txt" -n 15 #Search text files with the limit locate -c [.sh]* #Count the number of matches locate -i filename #Search a file in a case-insensitive manner find It is used to search and locate the files and folders based on the condition. Examples: find . -name filename.txt #Search a file in the current directory find . -iname filename.txt #Search a file case-insensitively find /temp -name filename.txt #Search the file in the particular folder find / -type d -name dirname #Search the directory in ‘/’ folder whereis It is used to search the location of the source, binary, and the manual page files for a command. Examples: whereis -l #Find the directories where the `whereis` command will search whereis bash #Find the location of the `bash` whereis man uptime #Find the location of `man` and `uptime` Network Information ping It is used to test a host is reachable or not in the network. Examples: ping www.google.com #Test using the URL address ping 142.250.199.36 #Test using IP address ping -c 5 www.google.com #Test will stop after sending 5 packets ping -i 3 www.google.com #Test using 3 seconds interval ifconfig It is used to configure, manage, and query network interface parameters by using a command-line interface or system configuration script. You have to install the net-tools package to use this command. Examples: ifconfig #Display all active interfaces ifconfig -a #Display all active or inactive interfaces ifconfig eth0 #Display information about the specific interface ifconfig eth0 up #Activate the particular interface ifconfig eth0 down #Deactivate the particular interface ifconfig eth0 172.16.25.125 #Set the IP address for the particular interface dig It is used to search the DNS name servers for DNS lookup. Examples: dig example.com #Find the IP address of the hostname dig 93.184.216.34 #Find the hostname of the IP address dig example.com MX #Find the MX of a hostname netstat It is used to get information about the network activities and display which ports are open or have established connections. It is a useful tool to debug network problems. Examples: netstat –a #Print all ports and connections netstat –at #Print all TCP ports netstat –au #Print all UDP ports netstat -l #Print all listening ports of all protocols ip It is used to bring the interface up or down, assign and remove the address, etc. Examples: ip addr show #Display information of all IP addresses ip addr show dev eth0 #Display information about a single interface sudo ip address add 192.168.10.50/24 dev eth0 #Assign IP to an interface Bash Shortcut Keys alt+ctrl+t It is used to open a new terminal window. ctrl+c It is used to stop the running command. ctrl+a It is used to go to the start of the line. ctrl+e It is used to go to the end of the line. ctrl+u It is used to cut the line before the cursor to the clipboard. ctrl+k It is used to cut the line after the cursor to the clipboard. ctrl+r It is used to search history. ctrl+z It is used to suspend the current foreground process. !! It is used to repeat the last command. !$ It is used to read the last argument of the previous command. !* It is used to read all arguments of the previous command. Bash Script Basics Variables The variable is used to store and read data. Example: website="linuxhint.com" #Assign a string value to a variable echo $website #Print the value of the variable echo "The website name is $website" #Print the variable with another string Length The length of a string can be calculated in bash in different ways. The simple way to count the length is shown below. Example: var="Hello World" #Assign String value echo ${#var} #Print the length of the sting Substring Sometimes it requires cutting the portion of a string that is called substring. Example: var="Hello World" #Assign String value echo "${var:0:5}" #Print the substring, ‘Hello’ echo "${var:6:5}" #Print the substring, ‘World’ Uppercase The string can be converted into uppercase by using `tr` command and ‘^^’ operator. Example: var="Hello" echo $var | tr [:lower:] [:upper:] #Convert to uppercase using `tr` echo ${var^^} #Convert to uppercase using '^^' Lowercase The string can be converted into lowercase by using `tr` command and ‘,,’ operator. Example: var="Hello" echo $var | tr [A-Z] [a-z] #Convert to lowercase using `tr` echo ${var,,} #Convert to lowercase using '^^' If Statement If statement is used to define the conditional statements. Example: str="file.txt" if [[ -f $str ]]; then #Check the file exists or not echo "$str is a file." elif [[ -d $str ]]; then #Check the directory exist or not echo "$str is a directory." else echo "$str is not a fir or directory." fi Case statement The case statement can be used as an alternative to if statement. Example: id=45 case $id in 67) echo "Group-1";; #Match with an ID 45 | 89) echo "Group-2";; #Match with any of two IDs 92 | 12 | 54) echo "Group-3" ;; #Match with any of three IDs *) echo "You are not selected." ;; esac For loop It is used to iterate some statements multiple times or read the array values. Example: for i in 23 67 45 11 89 #Iterate the loop for 5 times do echo "The value of i = $i" #Print each iteration value done While loop It is used as the alternative of for loop. Example: n=10 #Initialize a number while [ $n -le 20 ] #Iterate the loop until the value reaches to 20 do echo "n=$n" #Print the current value of $n n=$(( $n + 5 )) #Increment the value of $n by 5 done Function It is used to define a block of code with a name that can be executed anytime by calling the function name. Example: function_name() { #Define the function echo "Hello World" } echo $(function_name) #Call the function eval It is used to execute arguments like a shell command. Example: c1=$(( 10 + 15 )) #Define the argument c2="echo " #Define the second argument eval $c1 $c2 #Execute the arguments set It is used to control certain flags and features in Bash to set the behavior of the scripts. Examples: set –C #Disable the feature of overwriting the file set –e #Stop script immediately when an error occurs unset Ii is used to delete the variables during the program execution. It can be used to delete both functions and shell variables. Example: var="Hello World" #Assign String value echo $var #Print the assigned variable unset $var #Unset the variable echo $var #Print the variable after unset View the full article

Last year we wrote A Kubernetes User's Guide to HashiCorp Nomad — a resource for learning the equivalent terminologies, comparisons, and differentiations between HashiCorp Nomad and Kubernetes. As a follow up, we also wrote A Kubernetes User's Guide to HashiCorp Nomad Secret Management, which compared Kubernetes and Nomad’s integration workflows with HashiCorp Vault. This year, we’ve designed a companion cheat sheet that condenses some of the concept comparisons made in those guides and adds original content. This new, one-page reference PDF starts with a list of Kubernetes commands and their Nomad equivalents and has a list of Kubernetes concepts and their Nomad equivalents at the bottom. »The Cheat Sheet Below is an image of the Kubernetes to Nomad cheat sheet and a PDF download link (no registration required): »Contribute on the GitHub Page Along with the PDF cheat sheet, we’ve also started a GitHub page for collecting Nomad cheat sheets. The Kubernetes-to-Nomad commands and concepts cheat sheets are the first two resources to be added to this repository, which is now open for edits and additions by the community. So if you have your own set of common Kubernetes commands that you’ve translated into Nomad (or vice versa), or you think there’s an error that needs to be updated, you can easily put in a pull request on the GitHub page. And if you have an idea for another Nomad cheat sheet, and you’d like to submit some of it, go ahead and write a pull request for that as well. This cheat sheet and its various deployments were created by Michael Schurter, Kerim Satirli, Erik Veld, Taylor Dolezal, Jacquie Grindrod, Andrei Burd, Charlie Voiselle, Amier Chery, Kristopher Hughes, and Paul Burt. View the full article

Last year we wrote A Kubernetes User's Guide to HashiCorp Nomad — a resource for learning the equivalent terminologies, comparisons, and differentiations between HashiCorp Nomad and Kubernetes. As a follow up, we also wrote A Kubernetes User's Guide to HashiCorp Nomad Secret Management, which compared Kubernetes and Nomad’s integration workflows with HashiCorp Vault. This year, we’ve designed a companion cheat sheet that condenses some of the concept comparisons made in those guides and adds original content. This new, one-page reference PDF starts with a list of Kubernetes commands and their Nomad equivalents and has a list of Kubernetes concepts and their Nomad equivalents at the bottom. »The Cheat Sheet Below is an image of the Kubernetes to Nomad cheat sheet and a PDF download link (no registration required): »Contribute on the GitHub Page Along with the PDF cheat sheet, we’ve also started a GitHub page for collecting Nomad cheat sheets. The Kubernetes-to-Nomad commands and concepts cheat sheets are the first two resources to be added to this repository, which is now open for edits and additions by the community. So if you have your own set of common Kubernetes commands that you’ve translated into Nomad (or vice versa), or you think there’s an error that needs to be updated, you can easily put in a pull request on the GitHub page. And if you have an idea for another Nomad cheat sheet, and you’d like to submit some of it, go ahead and write a pull request for that as well. This cheat sheet and its various deployments were created by Michael Schurter, Kerim Satirli, Erik Veld, Taylor Dolezal, Jacquie Grindrod, Andrei Burd, Charlie Voiselle, Amier Chery, Kristopher Hughes, and Paul Burt. View the full article

Your product teams might ask - “Why does it take so long to build a feature or application?” Building applications is a heavy lift due to the technical complexity, which includes the complexity of backend services that are used to manage and store data. Every moment focused on this technical complexity is a distraction from delivering on core business value. Firestore alters this by having Google Cloud manage your backend complexity through a complete backend-as-a-service! Firestore is a serverless, NoSQL document database that unlocks application innovation with simplicity, speed and confidence. It acts as a glue that intelligently brings together the complete Google Cloud backend ecosystem, in-app services from Firebase and core UI frameworks & OS from Google. Click to enlarge What is Firestore? Firestore is a serverless, fully managed NoSQL document database that scales from zero to global scale without configuration or downtime. Here’s what makes Firestore unique: Ideal for rapid, flexible and scalable web and mobile development with direct connectivity to the database. Supports effortless real time data synchronization with changes in your database as they happen. Robust support for offline mode, so your users can keep interacting with your app even when the internet isn't available or is unreliable. Fully customizable security and data validation rules to ensure the data is always protected Built-in strong consistency, elastic scaling, high performance & best in class 99.999% availability Integration with Firebase and Google Cloud services like Cloud Functions and BigQuery, serverless data warehouse. In addition to a rich set of Google Cloud service integrations, Firestore also offers deep one-click integrations with a growing set of3rd party partners via Firebase Extensions to help you even more rapidly build applications. Document-model database Firestore is a Document-model database. All of your data is stored in “documents” and then “collections”. You can think of a document as a JSON object. It's a dictionary with a set of key-value mappings, where the values can be several different supported data types including strings, numbers or binary values.. These documents are stored in collections. Documents can't directly contain other documents, but they can point to subcollections that contain other documents, which can point to subcollections, and so on. This structure brings with it a number of advantages. For starters, all queries that you make are shallow, meaning that you can grab a document without worrying about grabbing all the data underneath it. And this means that you can structure your data hierarchically in a way that makes sense to you logically, without having to worry about grabbing tons of unnecessary data. How to use Firestore? Firestore can be used in two modes: Firestore in Native Mode: This mode is differentiated by its ability to directly connect your web & mobile app to Firestore. Native Mode supports up to 10K writes per second, and over a million connections. Firestore in Datastore Mode: This mode supports only server-side usage of Firestore, but supports unlimited scaling, including writes. ConclusionWhatever your application use case may be, if you want to build a feature or an application quickly using Firestore backend-as-a-service. For a more in-depth look into Firestore check out the documentation. For more #GCPSketchnote, follow the GitHub repo. For similar cloud content follow me on Twitter @pvergadia and keep an eye out on thecloudgirl.dev. Related Article Cloud Firestore explained: for users who never used Firestore before In this article, I will break down some database basics, terms you should know, what Firestore is, how it works, how it stores data, and ... Read Article

App Engine is a fully managed serverless compute option in Google Cloud that you can use to build and deploy low-latency, highly scalable applications. App Engine makes it easy to host and run your applications. It scales them from zero to planet scale without you having to manage infrastructure. App Engine is recommended for a wide variety of applications including web traffic that requires low-latency responses, web frameworks that support routes, HTTP methods, and APIs. Click to enlarge Environments App Engine offers two environments; here’s how to choose one for your application: App Engine Standard - Supports specific runtime environments where applications run in a sandbox. It is ideal for apps with sudden and extreme traffic spikes because it can scale from zero to many requests as needed. Applications deploy in a matter of seconds. If your required runtime is supported and it's an HTTP application, then App Engine Standard is the way to go.App Engine Flex - Is open and flexible and supports custom runtimes because the application instances run within Docker containers on Compute Engine. It is ideal for apps with consistent traffic and regular fluctuations because the instances scale from one to many. Along with HTTP applications it also supports applications requiring WebSockets. The max request timeout is 60 minutes. How does it work No matter which App Engine environment you choose, the app creation and deployment process is the same. First write your code, next specify the app.yaml file with runtime configuration, and finally deploy the app on App Engine using a single command: gcloud app deploy. Notable features Developer friendly - A fully managed environment lets you focus on code while App Engine manages infrastructure. Fast responses - App Engine integrates seamlessly with Memorystore for Redis enabling distributed in-memory data cache for your apps. Powerful application diagnostics - Cloud Monitoring and Cloud Logging help monitor the health and performance of your app and Cloud Debugger and Error Reporting help diagnose and fix bugs quickly. Application versioning - Easily host different versions of your app, and easily create development, test, staging, and production environments. Traffic splitting - Route incoming requests to different app versions forA/B tests incremental feature rollouts, and similar use cases. Application security - Helps safeguard your application by defining access rules with App Engine firewall and leverage managed SSL/TLS certificates by default on your custom domain at no additional cost. ConclusionWhether you need to build a modern web application or a scalable mobile backend App Engine has you covered. For a more in-depth look, check out the documentation. Click here for demos on how to use serverless technology and free hands-on training. App Engine in a minute For more #GCPSketchnote, follow the GitHub repo. For similar cloud content follow me on Twitter @pvergadia and keep an eye out on thecloudgirl.dev. Related Article Curious about Google Cloud Bare Metal Solution? Start here. Bare Metal solution helps you modernize your specialized Oracle workloads by providing an easier and a faster migration path while mainta... Read Article

If you want data as an ally to steer your business, you should look at Dataprep by Trifacta. Data is needed in every aspect of your business from optimizing profit margin, identifying new market opportunities, predicting next-best offer, responding to fraud, or reporting on regulatory obligations. Whether you are in a leadership position leveraging data for decision-making or in a support role to deliver clean and trustworthy data, you know how much effort (and often frustration) it takes to prepare data and make it ready for consumption. Where is the data? How do I access it? What’s in the data? Can I trust the data? How can I turn it into a format that I can use? How do I combine it with other sets of data? How do I automate all this process? Let’s see how you can turn this cumbersome and painful process into a seamless and fun experience with Dataprep! Click to enlarge What is data preparation? Data preparation, also known as data wrangling, is a self-service activity to access, assess, and convert disparate, raw, messy data into a clean and consistent view for your analytics and data science needs. Preparing data is critical but time-intensive; data teams spend up to 80% of their time converting raw data into high-quality, analysis-ready output. The work of preparing raw data into pristine asset consists of 6 iterative steps: Discover How do I access the data? What is in the data? Can I trust it? Dataprep helps you access diverse data sources and discover features of the data to quickly determine its value. With datatype inference, column-level profiles, interactive quality bars, and histograms, you get immediate visibility into trends and data issues, guiding your transformation process. Structure Can the data be used with its current format? Structuring refers to actions that change the form or schema of your data. Splitting columns, pivoting rows, and deleting fields are all forms of structuring. Dataprep predicts and suggests the next best transformation based on the data you’re working with and the type of interaction you applied to the data. Clean Are all the dates valid? Is the category correct? Is a SKU missing? During the cleaning stage, Dataprep surfaces data quality issues, such as missing or mismatched values, and suggests the appropriate transformations (formatting the date for you, normalizing the category into a consistent list, replacing or deleting the SKU) to correct these values from the dataset. Enrich Do you want to keep your data alone or augment it with other datasets from public or internal repositories? The data you need may be spread across multiple files, applications, and databases. To gather all the necessary insights, you need to enrich your existing dataset by joining and aggregating various data sources. With Dataprep you can quickly execute lookups to data dictionaries or perform joins and unions with disparate datasets. Validate Do you trust the transformations you applied to your data? Is this an accurate result? Dataprep profiles and provides data quality indicators across the complete transformed dataset. Here, you can do a final check for any data inaccuracy that wasn’t initially identified. Validate that your output dataset has the intended structure and content before publishing it for broader analysis. Publishing How should the resulting clean data output be consumed? At what frequency should it be refreshed? When your data has been successfully structured, cleaned, enriched, and validated, it’s time to publish your wrangled output for use in downstream analytics processes. Dataprep offers all you need to automate your data pipeline and publish it for your consuming analytics applications. Dataprep key features Built-in data quality assessment and validation As soon as you access data, Dataprep instantly discovers it and surfaces it in a familiar and intuitive grid interface. You trust what you see. Dataprep shows you value distribution, pattern formats, and highlights outliers. You even get suggestions to create data monitoring rules to track and resolve data quality issues. All this with visual interaction that reflects in real-time the transformations applied to the data. So you always see exactly what your data will look like after it is transformed. Automated data pipelines The data preparation recipes you author with Dataprep can be operationalized by data engineers or operation teams and controlled and monitored confidently. Dataprep’s end-to-end data pipeline leverages the auto-scaling processing capabilities of Dataflow & BigQuery to handle any volume of data–from Gigabytes to Petabytes. In addition, Dataprep integrates with Cloud IAM/OAuth for data security and peace of mind. And the magic is that you can just do it with clicks, not code (though coding is possible if that’s your preference!) Visualize & accelerate data transformation Ultimately, you want data ready quickly for your analysis, and you want to trust it. But there’s so much data and there are so many rules to apply! No worries, with Dataprep you’ll get ML-based suggestions to transform and clean the data. Select the one suggestion that is the most relevant or modify it for your needs. Standardize, structure, join, union, pivot, unpivot, aggregate, calculate any formula with a few clicks and get immediate feedback to your transformation. Data preparation becomes a game and you’ll enjoy it. The only regret: you may spend 90% less time preparing data now that you find it so fun! How does Dataprep work? Dataprep by Trifacta is a serverless and native Google Cloud data preparation solution as part of the broader Google Cloud Smart Analytics portfolio. Dataprep allows data analysts, business analysts, data engineers, and data scientists to visually explore, clean, and prepare big data. Dataprep connects to BigQuery, Cloud Storage, Google Sheets, and hundreds of other cloud applications and traditional databases so you can transform and clean any data you want. Dataprep is built on top of Dataflow and BigQuery. That means any data transformation and cleaning rules you design can easily scale to transform any data, big or small by translating Dataprep data transformation recipes into Dataflow jobs or BigQuery SQL statements. Once your data is ready in BigQuery or Cloud Storage, you can analyze it with Data Studio or Looker, train machine learning models with Vertex AI services, or get insight with other analytics partner solutions such as Qlik or Tableau. Dataprep, as a native service to Google Cloud, can be governed and automated via APIs to be controlled by Cloud Composer and Cloud Functions for example. Conclusion Enough words. Do you want to experience Dataprep first hand? This is easy, try this data warehouse design pattern leveraging Google Sheets, Dataprep, BigQuery, and Data Studio and get a sense of what a comprehensive and scalable analytics solution looks like. Related Article 5 cheat sheets to help you get started on your Google Cloud journey Whether you need to determine the best way to move to the cloud, or decide on the best storage option, we've built a number of cheat shee... Read Article