Problem Notebook or job execution stops and returns either of the following errors: Run result unavailable: job failed with error message Context ExecutionContextId(1731742567765160237) is disconnected. Can’t attach this notebook because the cluster has reached the attached notebook limit. Detach a notebook and retry. Cause When you attach a noteboo...

Problem Your job fails with a Java IndexOutOfBoundsException error message: java.lang.IndexOutOfBoundsException: index: 0, length: <number> (expected: range(0, 0)) When you review the stack trace you see something similar to this: Py4JJavaError: An error occurred while calling o617.count. : org.apache.spark.SparkException: Job aborted due to s...

Problem You are trying to create a Parquet table using TIMESTAMP, but you get an error message. Error in SQL statement: QueryExecutionException: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.UnsupportedOperationException: Parquet does not support timestamp. See HIVE-6384 Example code %sql CREATE EXTERN...

Problem You are using the Databricks Connect client with Databricks Runtime 6.4 and receive an error message which states that the client does not support the cluster. Caused by: java.lang.IllegalArgumentException: The cluster is running server version `dbr-6.4` but this client only supports Set(dbr-5.5). You can find a list of client releases at ht...

Problem You are trying to install the PyStan PyPi package on a Databricks Runtime 6.4 Extended Support cluster and get a ManagedLibraryInstallFailed error message. java.lang.RuntimeException: ManagedLibraryInstallFailed: org.apache.spark.SparkException: Process List(/databricks/python/bin/pip, install, pystan, --disable-pip-version-check) exited wit...

Problem You are trying to create a token on behalf of a service principal , using /2.0/token-management/on-behalf-of/tokens in the REST API but are getting a PERMISSION_DENIED error. { "error_code": "PERMISSION_DENIED", "message": "User xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx does not have permission to use tokens." } Cause This happens when the servic...

The majority of Databricks customers use production Databricks runtime releases (AWS | Azure | GCP) for their clusters. However, there may be certain times when you are asked to run a custom Databricks runtime after raising a support ticket. Warning Custom Databricks runtime images are created for specific, short-term fixes and edge cases. If a cust...

In this article we show you how to display detailed timestamps, including the date and time when a file was created or modified. Use ls command The simplest way to display file timestamps is to use the ls -lt <path> command in a bash shell. For example, this sample command displays basic timestamps for files and directories in the /dbfs/ folde...

Problem You are using a cluster running Databricks Runtime 7.3 LTS and above. You have enabled table access control for your workspace (AWS | Azure | GCP) as the admin user, and granted the SELECT privilege to a standard user-group that needs to access the tables. A user tries to access an object in the database and gets a SecurityException error me...

Problem You have imported Python libraries, but when you try to execute Python code in a notebook you get a repeating message as output. Example 1: INFO:py4j.java_gateway:Received command c on object id p0INFO:py4j.java_gateway:Received command c on object id p0INFO:py4j.java_gateway:Received command c on object id p0INFO:py4j.java_gateway:Received ...