updateClassifiers doesn't download sources and docs for updated snapshot dependencies #1750
Comments
|
for me even non-snapshot deps sources are not downloaded |
|
this also breaks gen-idea since it's no longer able to attach sources to libraries |
|
Snapshot dependencies sources are not downloaded at all for us (independently of cached resolution) |
|
can you try updateOptions := updateOptions.value.withCachedResolution(false).withLatestSnapshots(true) ? |
|
as I said, we already tried it. No way. log is (for example): although sources was uploaded too, of course. |
|
Well, it looks like it's broken since 0.13.2 or 0.13.1. 2014-12-14 16:37 GMT+03:00 eugene yokota [email protected]:
|
|
created #1776 |
|
I've been trying to reproduce this, but I can't, trying sbt 0.13.8, 0.13.7 and 0.13.6: // build.sbt
name := "t"
version := "1.0"
scalaVersion := "2.11.6"
resolvers += Resolver sonatypeRepo "snapshots"
libraryDependencies += "com.chuusai" %% "shapeless" % "2.2.1-SNAPSHOT"
@newffy Can you check if this bug is still present? And if so find a way to reproduce the problem with publicly available artifacts and repositories? |
|
It's all right when you download snapshot deps at first time. After that if 2015-06-07 19:22 GMT+03:00 Dale Wijnand [email protected]:
|
|
So seems like the way snapshot updates are handled is that the pom is downloaded and, if it's a newer pom, Ivy deletes the dependency jar from the Ivy cache, so then later it doesn't find it present, and re-downloads it. For some reason the Here's the code I'm talking about: Seeking Ivy help! |
|
The Compare <publications>
<artifact name="sbt-gh1750-bippy_2.11" type="jar" ext="jar" conf="master"/>
</publications>
<dependencies>
<dependency org="org.scala-lang" name="scala-library" rev="2.11.7" force="true"
conf="compile->compile(*),master(compile);runtime->runtime(*)"
/>
</dependencies> |
|
Interestingly invoking <?xml version="1.0" encoding="UTF-8"?>
<ivy-module version="2.0" xmlns:e="http://ant.apache.org/ivy/extra">
<info organisation="com.dwijnand" module="sbt-gh1750-bippy_2.11"
revision="0.1.0-SNAPSHOT" status="integration" publication="20150725004740"
>
<description>sbt-gh1750-bippy</description>
</info>
<configurations>
<conf name="compile" visibility="public" description=""/>
<conf name="runtime" visibility="public" description="" extends="compile"/>
<conf name="test" visibility="public" description="" extends="runtime"/>
<conf name="provided" visibility="public" description=""/>
<conf name="optional" visibility="public" description=""/>
<conf name="sources" visibility="public" description=""/>
<conf name="docs" visibility="public" description=""/>
<conf name="pom" visibility="public" description=""/>
</configurations>
<publications>
<artifact name="sbt-gh1750-bippy_2.11" type="pom" ext="pom" conf="pom"/>
<artifact name="sbt-gh1750-bippy_2.11" type="jar" ext="jar" conf="compile"/>
<artifact name="sbt-gh1750-bippy_2.11" type="src" ext="jar" conf="sources" e:classifier="sources"/>
<artifact name="sbt-gh1750-bippy_2.11" type="doc" ext="jar" conf="docs" e:classifier="javadoc"/>
</publications>
<dependencies>
<dependency org="org.scala-lang" name="scala-library" rev="2.11.7"
conf="compile->default(compile)"
/>
</dependencies>
</ivy-module> |
|
Going back a little, it seems the This leads me to git blame myself to d8b3118 and down the links that provides. Seems there's lots of things going on here, Ivy things, Maven things, sbt overriding ivy things, sbt changing ivy things in ivy, wrong POM things.. as well as performance concerns. So given this context I'm not sure how this bug should be solved. |
|
@dwijnand That's actually the desgin of sbt is to disable that feature. It turns out Ivy is issueing an http HEAD request to check existence of files before downloading. This is expensive, and it doesn't actually cache FAILURE (which maven does). Additionally, maven doesn't check existence of an artifact until it's asked for. SO, what we're trying to do in sbt is mimic the maven style, but use the Ivy interfaces/API. That's why this is so wrong. We have two options:
We really really want to fix 2, because we think we could eek a lot of performance out of Ivy if we fix 2. TO resolve this bug you only have to fix 1, which I think may be something a bit smalelr and more paletable. In any case, as I said in gitter, I'm still trying to think through (2) a bit more, as I'd like to nail a solution to that soon. |
|
@dwijnand It appears like the core issue here is related to the maven
I think what's happening is we're essentially asking for |
|
@jsuereth thanks for spending time looking into this, I truly need all the help I can get. I'm not sure if this is to do with unique version, as, at least on Nexus rather than Artifactory, the sources jar is available as I'm happy to look into how to fix 2, as I've been working on this exclusively for two months now, I don't mind trying to fix it the best way possible. But I wonder, if the desired result is, given the pom has a newer publication date, delete old jars so that the updated version can be downloaded, why not just try to delete any known, common classifier jars (like sources, srcs, javadoc) and then get them downloaded again. Is it better to have an outdated sources jar present if there's no updated sources jar available? But I guess this is the "fix 1" solution here. I'll try reproducing the performance issue with a missing sources jar, removing that sbt override, and see if I can come up with a satisfactory failure cache to mitigate. It's a shame it means working in two repos/builds. Again, thanks for your help. |
|
@dwijnand Anytime! I'm hoping I can help you out here eventually. Your idea to delete old jars when an updated version is downloaded actually sounds like it should work out as a "fix 1" workaround. COuld bear fruit. As far as fix #2, I'd really like to just redo how Ivy caches from the ground up, and rewrite EVERY dependency resolver to use the new caching mechanism. We might even be able to make that work :) |
|
I took some more detailed notes on the sequence of events, for the purposes of exploring both avenues, particularly so I can being to understand what things are called. It's quite a wash of detail, mostly for me, but might be interesting in general, particularly if anything is or looks wrong. |
|
See sbt/ivy#15. |
Force deleting sources/javadoc to fix sbt/sbt#1750
Hello.
sbt.version=0.13.7
we are using corporate maven repository (served by artifactory 3.4.1)
Our libraryDependencies:
during updateClassifiers only jars are downloaded, without sources and docs, although they all have published sources and docs. This is really annoying, I'm ready to supply all info you need.
We are using
for publishing
updateClassifiers log:
The text was updated successfully, but these errors were encountered: