0 Making your own OneDrive client to avoid shared quota? When GUI is unavailable, try the Azure CLI
Erudition edited this page 2024-09-24 00:03:24 +00:00

(This is mostly a personal experience report, but I captured the details so you can make this work with a lot less effort.) I ran into the onedrive quota with the shared key, so I was determined to make my own. In my case the GUI was blocked by my org, so I couldn't follow the instructions on the rclone website. image

That screen gave me the idea to try my hand at doing it with the terminal. Perhaps such instructions could be included in rclone as well. Better yet would be for rclone to do this with the graph api all on its own.

Create an application registration using Azure CLI

On my Guix gnu/linux machine it would be hard to install the az binary. But as the docs suggested I can sign up for a free trial of Azure portal to use Cloud Shell, which already has the az cli installed. So I did that, and tried hacking away. Reading the docs and seeing what worked.

Create command

az ad app create \
--display-name="$USER rclone" \
--sign-in-audience AzureADandPersonalMicrosoftAccount \
--enable-access-token-issuance true \
--enable-id-token-issuance true \
--web-redirect-uris "http://localhost:53682/" \
--web-home-page-url "https://rclone.org/onedrive/" \
--key-display-name rclonesecret \
--end-date '$(date -d'2 years')'

Later on I capture the output of this into a bash script, adding "--query appId --output tsv". You'll need the output for the app id, if you lose it you'll have to comb through the database.

Breakdown of arguments

--display-name=rclone

Can be anything, but required.

--sign-in-audience AzureADandPersonalMicrosoftAccount

As per rclone instructions to select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox) in the gui. For onedrive business the instructions say to use AzureADMyOrg instead which would translate to Accounts in this organizational directory only (*** - Single tenant).

Differences can be seen here.

Web arguments

--enable-access-token-issuance true (and) --enable-id-token-issuance true

Enable OAuth flow so we can sign in with the website. May not be required, haven't found the default. One is for ID tokens, the other for access tokens. perhaps only access tokens are needed?

--web-redirect-uris "http://localhost:53682/"

This one is important so the oauth portal can be passed back to your local rclone.

--web-home-page-url "https://rclone.org/onedrive/"

Not required. Different from the redirect - this is just where microsoft would point you as the "home page" for the application.

Key Credentials

--key-display-name rclonesecret

Name the new client secret key.

--end-date '2027-01-01'

As per rclone instructions, sets the expiration date for the client secret. (e.g. '2017-12-31T11:59:59+00:00' or '2017-12-31') There is also --start-date which takes the same format, if you don't want it to start working right away for some reason.

Should not be a Public app

--is-fallback-public-client false

Not sure if this is needed, but seems it would enhance compatibility from different systems. Edit: oh no it actually should be false. "public" applications are not allowed to pass secrets. Normally it detects it automatically but I added "mobile and desktop" platforms besides just "web" and that made it public apparently, so rclone which failed with the message:

{"error":"invalid_client","error_description":"AADSTS700025: Client is public so neither 'client_assertion' nor 'client_secret' should be presented. Trace ID: 0001919c-b796-470e-b3ae-2a43d2990d00 Correlation ID: 130aa766-d7f1-4699-b5ac-9a7adb04e315 Timestamp: 2024-09-23 22:51:01Z","error_codes":[700025],"timestamp":"2024-09-23 22:51:01Z","trace_id":"0001919c-b796-470e-b3ae-2a43d2990d00","correlation_id":"130aa766-d7f1-4699-b5ac-9a7adb04e315"}

API says "Specifies the fallback application type as public client, such as an installed application running on a mobile device. The default value is false, which means the fallback application type is confidential client such as web app. There are certain scenarios where Microsoft Entra ID cannot determine the client application type (for example, ROPC flow where it is configured without specifying a redirect URI). In those cases, Microsoft Entra ID will interpret the application type based on the value of this property."

API Call with JSON

The rest needs to be done with json, which is tricky. You can see what I sent in the script below. Much of this is not required, but I wanted it to be presentable if an administrator passed over it...

"adminConsentDescription"

A description of the delegated permissions, intended to be read by an administrator granting the permission on behalf of all users. This text appears in tenant-wide admin consent experiences.

"adminConsentDisplayName"

The permission's title, intended to be read by an administrator granting the permission on behalf of all users.

"id"

Supply your own generated uuid for the scope.

"isEnabled"

if we don't set this, apps start with their sopes disabled

"type" user

Specifies whether this delegated permission should be considered safe for non-admin users to consent to on behalf of themselves, or whether an administrator consent should always be required. While Microsoft Graph defines the default consent requirement for each permission, the tenant administrator may override the behavior in their organization (by allowing, restricting, or limiting user consent to this delegated permission).

"userConsentDescription"

A description of the delegated permissions, intended to be read by a user granting the permission on their own behalf. This text appears in consent experiences where the user is consenting only on behalf of themselves.

"userConsentDisplayName"

A title for the permission, intended to be read by a user granting the permission on their own behalf. This text appears in consent experiences where the user is consenting only on behalf of themselves.

"preAuthorizedApplications"

I attempted to pre-authorize rclone. Not sure if it worked.

Full script

The last part with json gave me a permission error, meaning it probably had to be done some other way, I can't just overwrite the api json the way an administrator could. I needed to set the permission scopes somehow, or so I thought.

But Here is everything done so far in a reusable script:


# create app registration and extract appId
clientid=$(az ad app create --display-name='rclone' --sign-in-audience AzureADandPersonalMicrosoftAccount --enable-access-token-issuance true --enable-id-token-issuance true --web-redirect-uris 'http://localhost:53682/' --web-home-page-url 'https://rclone.org/onedrive/' --key-display-name rclonesecret --end-date "$(date -d'2 years')" --is-fallback-public-client false --query appId --output tsv)

# generate a UUID for the scope
uuid=$(uuidgen)

# set the API object as a JSON object
api=$(echo '{
    "acceptMappedClaims": null,
    "knownClientApplications": [],
    "oauth2PermissionScopes": [{
        "adminConsentDescription": "Allows desktop sync to Onedrive using a tool called rclone, which is often the only option for GNU/Linux users where OneDrive Desktop is unavailable. This Azure application must be created by the user to grant oAuth access to rclone due to the way OneDrive works with quota limits.",
        "adminConsentDisplayName": "OneDrive Desktop sync for GNU/Linux",
        "id": "'$uuid'",
        "isEnabled": true,
        "type": "User",
        "userConsentDescription": "Grants file access to your rclone instance!",
        "userConsentDisplayName": "'$USER' rclone",
        "value": "'$clientid'/.default"
    }],
    "preAuthorizedApplications": [{ "appId": "b15665d9-eda6-4092-8539-0eec376afd59", "delegatedPermissionIds": ["'$uuid'"]}],
    "requestedAccessTokenVersion": 2
}' | jq .)


# Update app registration with App ID URL and api object
az ad app update \
    --id $clientid \
    --identifier-uris api://$clientid \
    --set api="$api"

Inspired by StackOverflow answer I learned from https://learn.microsoft.com/en-us/graph/api/resources/application?view=graph-rest-1.0

Back to the GUI (Bypassing the "you do not have access" screen)

At this point I had an appID, which flashes for a brief second on the gui before the block covers it. This means I could use web dev skills to inspect the underlying elements since they're still there.

It worked! I'm now using rclone with my own appid, without trying to get help from the IT department on campus.

Which is good for me finally finishing this incredibly long project, but bad for anyone who wants those final steps in script form. I stopped once I got it working. But I provide an example of the state of the app json before and after using the GUI so you can continue where I left off, knowing what shape it has to take. Use the az command manual.

Shortcuts to app screens

If you're in my situation, The app overview page is blocked, but the subpages are not. You can use these links to go directly to them! Replace the appID a70dcdfe-a7c4-4af5-8e5f-31f3ad393a86 with your own. Credentials page: https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps/CredentialsBlade/appId/a70dcdfe-a7c4-4af5-8e5f-31f3ad393a86/isMSAApp/ Authentication/Platform Configurations page: https://portal.azure.com/#blade/Microsoft_AAD_RegisteredApps/AuthenticationV2Blade/appId/a70dcdfe-a7c4-4af5-8e5f-31f3ad393a86/isMSAApp/ Web API page: https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps/WebApiBlade/appId/a70dcdfe-a7c4-4af5-8e5f-31f3ad393a86/isMSAApp/ Branding page: https://portal.azure.com/#blade/Microsoft_AAD_RegisteredApps/BrandingBlade/appId/a70dcdfe-a7c4-4af5-8e5f-31f3ad393a86/isMSAApp/

If we used the graph API instead

  • rclone logo can be added without gui
  • auth can be done natively in rclone without the browser.
  • probably can be all done in one step

Unregistered publisher problem

This should be a problem, but hasn't been: Starting November 9th, 2020 end users will no longer be able to grant consent to newly registered multitenant apps without verified publishers. Associate a verified Microsoft Partner Center (MPN) account with your application. A verified badge will appear in various places, including the application consent screen.
Add MPN ID to verify publisher The application publisher domain is set to mycompany.onmicrosoft.com, but onmicrosoft.com publisher domains are not allowed. Please use a custom domain in order to proceed. Note: this domain must be a DNS verified domain on the tenant and match the primary contact domain for your MPN account.

App Object before/after GUI

Before

{
    "addIns": [],
    "api": {
      "acceptMappedClaims": null,
      "knownClientApplications": [],
      "oauth2PermissionScopes": [],
      "preAuthorizedApplications": [],
      "requestedAccessTokenVersion": 2
    },
    "appId": "a70dcdfe-a7c4-4af5-8e5f-31f3ad393a86",
    "appRoles": [],
    "applicationTemplateId": null,
    "certification": null,
    "createdDateTime": "2024-09-23T20:42:36Z",
    "defaultRedirectUri": null,
    "deletedDateTime": null,
    "description": null,
    "disabledByMicrosoftStatus": null,
    "displayName": "rclone",
    "groupMembershipClaims": null,
    "id": "e200819a-a511-4a2a-ae3c-d2eba6a0d04a",
    "identifierUris": [],
    "info": {
      "logoUrl": null,
      "marketingUrl": null,
      "privacyStatementUrl": null,
      "supportUrl": null,
      "termsOfServiceUrl": null
    },
    "isDeviceOnlyAuthSupported": null,
    "isFallbackPublicClient": true,
    "keyCredentials": [],
    "nativeAuthenticationApisEnabled": null,
    "notes": null,
    "optionalClaims": null,
    "parentalControlSettings": {
      "countriesBlockedForMinors": [],
      "legalAgeGroupRule": "Allow"
    },
    "passwordCredentials": [],
    "publicClient": {
      "redirectUris": []
    },
    "publisherDomain": "mycompany.onmicrosoft.com",
    "requestSignatureVerification": null,
    "requiredResourceAccess": [],
    "samlMetadataUrl": null,
    "serviceManagementReference": null,
    "servicePrincipalLockConfiguration": null,
    "signInAudience": "AzureADandPersonalMicrosoftAccount",
    "spa": {
      "redirectUris": []
    },
    "tags": [],
    "tokenEncryptionKeyId": null,
    "uniqueName": null,
    "verifiedPublisher": {
      "addedDateTime": null,
      "displayName": null,
      "verifiedPublisherId": null
    },
    "web": {
      "homePageUrl": "https://rclone.org/onedrive/",
      "implicitGrantSettings": {
        "enableAccessTokenIssuance": true,
        "enableIdTokenIssuance": true
      },
      "logoutUrl": null,
      "redirectUriSettings": [
        {
          "index": null,
          "uri": "http://localhost:53682/"
        }
      ],
      "redirectUris": [
        "http://localhost:53682/"
      ]
    }
  }

After

{
  "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#applications/$entity",
  "addIns": [],
  "api": {
    "acceptMappedClaims": null,
    "knownClientApplications": [],
    "oauth2PermissionScopes": [],
    "preAuthorizedApplications": [],
    "requestedAccessTokenVersion": 2
  },
  "appId": "a70dcdfe-a7c4-4af5-8e5f-31f3ad393a86",
  "appRoles": [],
  "applicationTemplateId": null,
  "certification": null,
  "createdDateTime": "2024-09-23T20:42:36Z",
  "defaultRedirectUri": null,
  "deletedDateTime": null,
  "description": null,
  "disabledByMicrosoftStatus": null,
  "displayName": "rclone",
  "groupMembershipClaims": null,
  "id": "e200819a-a511-4a2a-ae3c-d2eba6a0d04a",
  "identifierUris": [
    "api://a70dcdfe-a7c4-4af5-8e5f-31f3ad393a86"
  ],
  "info": {
    "logoUrl": "https://aadcdn.msftauthimages.net/dbd5a2dd-3043yiiumsdt-cnblanxrohrijw6rniss02qncvedne/appbranding/gz5qrxfu4l4ovgnxo8gpilxhvwz1kxom5c4z-vxpsx0/1033/bannerlogo?ts=638627257905844375",
    "marketingUrl": null,
    "privacyStatementUrl": null,
    "supportUrl": null,
    "termsOfServiceUrl": null
  },
  "isDeviceOnlyAuthSupported": null,
  "isFallbackPublicClient": true,
  "keyCredentials": [],
  "nativeAuthenticationApisEnabled": null,
  "notes": null,
  "optionalClaims": null,
  "parentalControlSettings": {
    "countriesBlockedForMinors": [],
    "legalAgeGroupRule": "Allow"
  },
  "passwordCredentials": [
    {
      "customKeyIdentifier": null,
      "displayName": "rclone-secret",
      "endDateTime": "2026-09-23T22:17:02.418Z",
      "hint": "DJY",
      "keyId": "d350b28e-b1cd-46d9-b77f-37b9bafe4346",
      "secretText": null,
      "startDateTime": "2024-09-23T22:17:02.418Z"
    }
  ],
  "publicClient": {
    "redirectUris": []
  },
  "publisherDomain": "mycompany.onmicrosoft.com",
  "requestSignatureVerification": null,
  "requiredResourceAccess": [],
  "samlMetadataUrl": null,
  "serviceManagementReference": null,
  "servicePrincipalLockConfiguration": null,
  "signInAudience": "AzureADandPersonalMicrosoftAccount",
  "spa": {
    "redirectUris": []
  },
  "tags": [],
  "tokenEncryptionKeyId": null,
  "uniqueName": null,
  "verifiedPublisher": {
    "addedDateTime": null,
    "displayName": null,
    "verifiedPublisherId": null
  },
  "web": {
    "homePageUrl": "https://rclone.org/onedrive/",
    "implicitGrantSettings": {
      "enableAccessTokenIssuance": true,
      "enableIdTokenIssuance": true
    },
    "logoutUrl": null,
    "redirectUriSettings": [
      {
        "index": null,
        "uri": "http://localhost:53682/"
      }
    ],
    "redirectUris": [
      "http://localhost:53682/"
    ]
  }
}

Hope this helps someone, if not those watching than those who discover this page through a search engine.

As far as whether this Github issue is actionable, I'd love for the graph api to be used to do this for me -- but otherwise, adding CLI steps to the documentation is my request. As you can see, blocked from the Azure App Registrations portal does not mean unable to get the job done.