Skip to content

Commit

Permalink
Merge pull request #23 from Akianonymus/master
Browse files Browse the repository at this point in the history
Use drive api v3 | Parallel Uploading | Sub folder upload | Improve logging | Pass shellcheck | Migrate to getopts | Share file/folder options | Format code
  • Loading branch information
labbots committed Apr 17, 2020
2 parents 90bfcdc + 335c21e commit 0d67bae
Show file tree
Hide file tree
Showing 4 changed files with 862 additions and 334 deletions.
79 changes: 48 additions & 31 deletions README.md
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,44 +1,63 @@
**Google drive upload**
-------------------
Google drive upload is a Bash scripts based on v2 google APIs to upload files/directories into google drive. This is a minimalistic shell script which utilizes google OAuth2.0 device flow to generate access tokens to authorize application to upload files to your google drive.
# Google drive upload

Google drive upload is a Bash script based on v3 google APIs to upload files/directories into google drive. This is a minimalistic shell script which utilizes google OAuth2.0 device flow to generate access tokens to authorize application to upload files to your google drive.

Further usage documentation can be found at my blog page [Labbots.com](https://labbots.com/google-drive-upload-bash-script/ "Labbots.com").

**Dependencies**
----------------
## Dependencies

This script does not have very many dependencies. Most of the dependencies are available by default in most linux platforms. This script requires the following packages

- Curl
- sed (Stream editor)
- find command
- awk
- getopt
- Curl
- sed (Stream editor)
- find command
- awk
- getopts ( bash builtin )
- xargs

## Features

- No dependencies at all.
- Upload files and folders.
- Upload files in parallel.
- Upload sub-folders and content inside it hierarchically.
- Config file support ( easy to use script on multiple machines ).
- Uses latest gdrive v3 api.
- Share files after uploading.
- Pretty Logging.

**Usage**
-----
When the script is executed for the first time. It asks for few configuration variables interactively to connect with google APIs. The script requires Client Id and Client secret to access the APIs which can be generated at [google console].
Script also asks for root folder to be set to which the script uploads documents by default. The default folder will be the root of your google drive. If you want to upload documents to any specific directory by default then provide the google folder id of that folder which root folder id is requested. The google folder id can be found from the URL of the google drive folder.
## Usage

For example
>https://drive.google.com/drive/folders/8ZzWiO_pMAtBrLpARBZJNV09TVmM
When the script is executed for the first time. It asks for few configuration variables interactively to connect with google APIs. The script requires Client ID and Client secret to access the APIs which can be generated at [google console].
Script also asks for root folder to be set to which the script uploads documents by default. The default folder will be the root of your google drive. If you want to upload documents to any specific directory by default then provide the google folder ID or URL of that folder when root folder ID/URL is requested.

For example
`https://drive.google.com/drive/folders/8ZzWiO_pMAtBrLpARBZJNV09TVmM`

In the above URL. The folder id is ***8ZzWiO_pMAtBrLpARBZJNV09TVmM***

Note: You can input either URL or ID.

The default configurations of the script are store under **$HOME/.googledrive.conf**

The script can be used in the following way

./upload.sh <filepath> <foldername>
./upload.sh <filepath/folderpath> <foldername>
Above command will create a folder under the pre-configured root directory and upload the specified file under the folder name. If the folder already exists then the file is uploaded under the folder.

Other Options available are

-C | --create-dir <foldername> - option to create directory. Will provide folder id.
-r | --root-dir <google_folderid> - google folder id to which the file/directory to upload.
-v | --verbose - Display detailed message.
-h | --help - Display usage instructions.
-z | --config - Override default config file with custom config file.
-g | --globoff - Turn off URL globbing parser to allow filenames with literal [] and/or {}.
-r | --root-dir <google_folderid> - google folder id to which the file/directory to upload.
-s | --skip-subdirs - Skip creation of sub folders and upload all files inside the INPUT folder/sub-folders in the INPUT folder, use this along with -p/--parallel option to speed up the uploads.
-p | --parallel <no_of_files_to_parallely_upload> - Upload multiple files in parallel, only works along with --skip-subdirs/-s option, Max value = 10, low value are recommended.
-S | --share - Share the uploaded input file/folder, grant reader permission to the everyone with the link.
-v | --verbose - Display detailed message.
-V | --verbose-progress - Display detailed message and detailed upload progress( curl normal progress info ).
-i | --save-info <file_to_save_info> - Save uploaded files info to the given filename."
-h | --help - Display usage instructions.
-z | --config - Override default config file with custom config file.
-D | --debug - Display script command trace."

To create a folder:

Expand All @@ -47,20 +66,18 @@ This will give the folder id of the newly created folder which can be used to up

To Upload file to specific google folder

./upload.sh -v -r <google-folder-id> <file/directory-path>
./upload.sh -r <google-folder-id> <file/directory-path>

The script also allows to upload directories. If directory path is provided as argument to the script then the script recursively uploads all the files in the folder to google drive.
The script also allows to upload directories. If directory path is provided as argument to the script then the script recursively uploads all the sub-folder and files present in the heirarchial way as it is present on the local machine.

## Inspired By

**Inspired By**
----
* [github-bashutils] - soulseekah/bash-utils
* [deanet-gist] - Uploading File into Google Drive
- [github-bashutils] - soulseekah/bash-utils
- [deanet-gist] - Uploading File into Google Drive

**License**
----
MIT
## License

MIT

[github-bashutils]: <https://github.com/soulseekah/bash-utils>
[deanet-gist]:<https://gist.github.com/deanet/3427090>
Expand Down
104 changes: 66 additions & 38 deletions google-oauth2.sh
Original file line number Diff line number Diff line change
@@ -1,66 +1,94 @@
#!/bin/bash
#!/usr/bin/env bash

# A simple cURL OAuth2 authenticator
# A simple curl OAuth2 authenticator
#
# Usage:
# ./google-oauth2.sh create - authenticates a user
# ./google-oauth2.sh refresh <token> - gets a new token
#
# Set CLIENT_ID and CLIENT_SECRET and SCOPE
# See SCOPES at https://developers.google.com/identity/protocols/oauth2/scopes#docsv1

#!/bin/bashset -e
short_help() {
echo -e "\nNo valid arguments provided."
echo -e "Usage:\n"
echo -e " ./google-oauth2.sh create - authenticates a user."
echo -e " ./google-oauth2.sh refresh - gets a new access token."
exit 0
}

[ "$#" = "0" ] && short_help

# Clear nth no. of line to the beginning of the line.
clear_line() {
echo -en "\033[""$1""A"
echo -en "\033[2K"
}

[ "$1" = create ] || [ "$1" = refresh ] || short_help

echo "Starting script.."

CLIENT_ID=""
CLIENT_SECRET=""
SCOPE=${SCOPE:-"https://docs.google.com/feeds"}
SCOPE="https://www.googleapis.com/auth/drive"
REDIRECT_URI="urn:ietf:wg:oauth:2.0:oob"

if [ -e $HOME/.googledrive.conf ]
then
. $HOME/.googledrive.conf
fi
# shellcheck source=/dev/null
[ -e "$HOME"/.googledrive.conf ] && source "$HOME"/.googledrive.conf

echo "Checking credentials.."

if [ -z "$CLIENT_ID" ]
then
read -p "Client ID: " CLIENT_ID
if [ -z "$CLIENT_ID" ]; then
read -r -p "Client ID: " CLIENT_ID
unset token
echo "CLIENT_ID=$CLIENT_ID" >> $HOME/.googledrive.conf
echo "CLIENT_ID=$CLIENT_ID" >> "$HOME"/.googledrive.conf
fi

if [ -z "$CLIENT_SECRET" ]
then
read -p "Client Secret: " CLIENT_SECRET
if [ -z "$CLIENT_SECRET" ]; then
read -r -p "Client Secret: " CLIENT_SECRET
unset token
echo "CLIENT_SECRET=$CLIENT_SECRET" >> $HOME/.googledrive.conf
echo "CLIENT_SECRET=$CLIENT_SECRET" >> "$HOME"/.googledrive.conf
fi

sleep 1
clear_line 1
clear_line 1
echo "Required credentials set."
sleep 1

# Method to extract data from json response
function jsonValue() {
KEY=$1
num=$2
awk -F"[:,}][^:\/\/]" '{for(i=1;i<=NF;i++){if($i~/\042'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -n ${num}p | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' -e 's/[,]*$//'
jsonValue() {
num="$2"
grep \""$1"\" | sed "s/\:/\n/" | grep -v \""$1"\" | sed -e "s/\"\,//g" -e 's/["]*$//' -e 's/[,]*$//' -e 's/^[ \t]*//' -e s/\"// | sed -n "${num}"p
}

if [ "$1" == "create" ]; then
RESPONSE=`curl --silent "https://accounts.google.com/o/oauth2/device/code" --data "client_id=$CLIENT_ID&scope=$SCOPE"`
DEVICE_CODE=`echo "$RESPONSE" | jsonValue device_code`
USER_CODE=`echo "$RESPONSE" | jsonValue user_code`
URL=`echo "$RESPONSE" | jsonValue verification_url`
echo "\nVisit the below URL, tap on allow and then enter the code obtained:"
sleep 1
URL="https://accounts.google.com/o/oauth2/auth?client_id=$CLIENT_ID&redirect_uri=$REDIRECT_URI&scope=$SCOPE&response_type=code&prompt=consent"
echo -e """$URL""\n"
read -r -p "Enter the authorization code: " CODE

echo -n "Go to $URL and enter $USER_CODE to grant access to this application. Hit enter when done..."
read
CODE="$(echo "$CODE" | tr -d ' ' | tr -d '[:blank:]' | tr -d '[:space:]')"
if [ -n "$CODE" ]; then
RESPONSE="$(curl -s --request POST --data "code=$CODE&client_id=$CLIENT_ID&client_secret=$CLIENT_SECRET&redirect_uri=$REDIRECT_URI&grant_type=authorization_code" https://accounts.google.com/o/oauth2/token)"

RESPONSE=`curl --silent "https://accounts.google.com/o/oauth2/token" --data "client_id=$CLIENT_ID&client_secret=$CLIENT_SECRET&code=$DEVICE_CODE&grant_type=http://oauth.net/grant_type/device/1.0"`
ACCESS_TOKEN="$(echo "$RESPONSE" | jsonValue access_token)"
REFRESH_TOKEN="$(echo "$RESPONSE" | jsonValue refresh_token)"

ACCESS_TOKEN=`echo "$RESPONSE" | jsonValue access_token`
REFRESH_TOKEN=`echo "$RESPONSE" | jsonValue refresh_token`

echo "Access Token: $ACCESS_TOKEN"
echo "Refresh Token: $REFRESH_TOKEN"
echo "Access Token: $ACCESS_TOKEN"
echo "Refresh Token: $REFRESH_TOKEN"
else
echo -e "\nNo code provided, run the script and try again"
exit 1
fi
elif [ "$1" == "refresh" ]; then
REFRESH_TOKEN=$2
RESPONSE=`curl --silent "https://accounts.google.com/o/oauth2/token" --data "client_id=$CLIENT_ID&client_secret=$CLIENT_SECRET&refresh_token=$REFRESH_TOKEN&grant_type=refresh_token"`

ACCESS_TOKEN=`echo $RESPONSE | jsonValue access_token`

echo "Access Token: $ACCESS_TOKEN"
if [ -n "$REFRESH_TOKEN" ]; then
RESPONSE="$(curl -s --request POST --data "client_id=$CLIENT_ID&client_secret=$CLIENT_SECRET&refresh_token=$REFRESH_TOKEN&grant_type=refresh_token" https://accounts.google.com/o/oauth2/token)"
ACCESS_TOKEN="$(echo "$RESPONSE" | jsonValue access_token)"
echo "Access Token: $ACCESS_TOKEN"
else
echo "Refresh Token not set, use $0 create to generate one."
fi
fi

0 comments on commit 0d67bae

Please sign in to comment.