I try "python -m spacy download rocorenews_md", but Databricks is stupid and rasies…
- User
I try |"python -m spacy download ro_core_news_md", but Databricks is stupid and rasies error: requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /explosion/spacy-models/master/compatibility.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7fcda56136d0>, 'Connection to 10.241.155.1 timed out. (connect timeout=None)'))
- Assistant
When encountering a connection timeout error while trying to download a spaCy model in a Databricks environment, there are a few possible solutions you can try:
-
Network Configuration: Ensure that your Databricks workspace has access to the internet. If you're in a restricted network environment, you might need to configure network settings or use a proxy to allow outbound connections to the internet.
-
Use a Direct URL: If downloading directly from the command line doesn’t work due to network restrictions, you can manually download the model file from a local machine and then upload it to Databricks. Here are the steps:
- Download the model package from the spaCy models repository on a local machine.
- Upload the downloaded model package to DBFS (Databricks File System) or directly to your cluster.
- Install the model using pip by pointing to the uploaded file, e.g.,
!pip install /dbfs/path/to/ro_core_news_md-X.X.X.tar.gz.
-
Use an Alternative Host: If the timeout is specifically due to GitHub, consider using an alternative host or mirror to download spaCy models if available.
-
Check Firewall or Security Rules: If your Databricks environment is within a Virtual Private Cloud (VPC) or private subnet, ensure that the security group or firewall rules allow egress traffic to the GitHub servers or the public internet.
-
Increase Timeout: Although less likely to solve the fundamental issue, you can attempt to increase the timeout for connections in your Python script, though spaCy and requests don’t directly expose an easy way to do this without modifying library code.
-
Use a Different Method: As a last resort, consider using a different platform or method for running your code, where downloading models is not restricted, if none of the above methods work.
If the issue persists, reaching out to your network admin or Databricks support might be necessary to resolve network access issues.
-