Solution to failed to acquire username/password from local configuration on rust git libs

Let's say you need to work on a Rust project that depends on other libraries hosted in a private Git repository. This project uses the repository's HTTPS URL. For example:

your-lib = { git = "ssh://git@github.com/your-company/your-lib.git"}

Then you try to start the project and encounter the following error:

$ cargo run

    Blocking waiting for file lock on package cache

    Updating crates.io index

    Updating git repository `https://github.com/your-company/your-lib.git`

error: failed to get `outbox-pattern-processor` as a dependency of package `anti-fraud-service v0.1.0 (/project/your-project)`

Caused by:

  failed to load source for dependency `your-lib`

Caused by:

  Unable to update https://github.com/your-company/your-lib.git?tag=v1.0.0#49efa247

Caused by:

  failed to fetch into: /home/tiago.motta/.cargo/git/db/your-lib-31d59910066df49a

Caused by:

  revision 49efa2476e2613cf809315b4b0abf16f079b5dcb not found

Caused by:

  failed to authenticate when downloading repository

  * attempted to find username/password via git's `credential.helper` support, but failed

  if the git CLI succeeds then `net.git-fetch-with-cli` may help here

  https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli

Caused by:

  failed to acquire username/password from local configuration

The solution is simple. Just edit your global .gitconfig to replace HTTPS URLs with SSH by adding the following lines:

[url "ssh://git@github.com/"]

insteadOf = https://github.com/

What is the best rust lib to post custom metric on datadog?

The best library for recording custom metrics in Datadog using Rust is dogstatsd (https://crates.io/crates/dogstatsd).

The alternative statsd (https://crates.io/crates/statsd) does not allow sending tags. There is even a fork, datadog-statsd (https://crates.io/crates/datadog-statsd), that enables tag sending. However, both define the client without deriving Clone, which prevents using this client as with_state in Axum, resulting in an error similar to this:

the trait bound `...` is not satisfied

the trait `Clone` is not implemented for `...`rustcClick for full compiler diagnostic

method_routing.rs(168, 16): required by a bound in `post`

....rs(6, 1): consider annotating `...` with `#[derive(Clone)]`: `#[derive(Clone)]

`

the trait bound `...` is not satisfied

the trait `Clone` is not implemented for `...`rustcClick for full compiler diagnostic

method_routing.rs(168, 16): required by a bound in `post`

....rs(6, 1): consider annotating `...` with `#[derive(Clone)]`: `#[derive(Clone)]

`

the trait bound `fn(axum::extract::State<...>, Json<...>) -> impl futures::Future<Output = Result<hyper::Response<axum::body::Body>, AppError>> {...::handler}: Handler<_, _>` is not satisfied

the following other types implement trait `Handler<T, S>`:

  `Layered<L, H, T, S>` implements `Handler<T, S>`

  `MethodRouter<S>` implements `Handler<(), S>`rustcClick for full compiler diagnostic

....rs(55, 32): required by a bound introduced by this call

method_routing.rs(166, 16): required by a bound in `post`

the trait bound `fn(axum::extract::State<...>, axum::Json<...>) -> impl futures::Future<Output = Result<hyper::Response<axum::body::Body>, ...>> {...}: Handler<_, _>` is not satisfied

the following other types implement trait `Handler<T, S>`:

  `Layered<L, H, T, S>` implements `Handler<T, S>`

  `MethodRouter<S>` implements `Handler<(), S>`rustcClick for full compiler diagnostic

....rs(55, 32): required by a bound introduced by this call

method_routing.rs(166, 16): required by a bound in `post`

axum::routing::method_routing

pub fn post<H, T, S>(handler: H) -> MethodRouter<S, Infallible>

where

    H: Handler<T, S>,

    T: 'static,

    S: Clone + Send + Sync + 'static,

H = fn handler(State<...>, …) -> …, S = ...

 

Solution to failed to run custom build command for `openssl-sys v0.9.104`

 When upgrading "reqwests" library I got the following error on docker build:

#18 182.8 The following warnings were emitted during compilation:

#18 182.8 

#18 182.8 warning: openssl-sys@0.9.104: Could not find directory of OpenSSL installation, and this `-sys` crate cannot proceed without this knowledge. If OpenSSL is installed and this crate had trouble finding it,  you can set the `OPENSSL_DIR` environment variable for the compilation process. See stderr section below for further information.

#18 182.8 

#18 182.8 error: failed to run custom build command for `openssl-sys v0.9.104`

#18 182.8 

#18 182.8 Caused by:

#18 182.8   process didn't exit successfully: `/usr/app/target/release/build/openssl-sys-ff7852766e78b685/build-script-main` (exit status: 101)

#18 182.8   --- stdout

#18 182.8   cargo:rustc-check-cfg=cfg(osslconf...

 

Could not find the solution after googling, but I tried the obvious solution, install openssl-dev into the docker image:

FROM rust:1.82-alpine AS builder

RUN apk add --no-cache openssl-dev

After this I got the following error:

x86_64-alpine-linux-musl/bin/ld: cannot find -lssl: No such file or directory

x86_64-alpine-linux-musl/bin/ld: cannot find -lcrypto: No such file or directory

And the fix for this was adding the openssl-libs-static:

FROM rust:1.82-alpine AS builder

RUN apk add --no-cache openssl-dev openssl-libs-static

 

Fixed

Solution to cargo-tarpaulin and missing `GLIBC_2.38' not found

Using cargo-tarpaulin 0.31.4 using binstall on Ubuntu 22.04 may result in errors like the following:

cargo-tarpaulin: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by cargo-tarpaulin)

cargo-tarpaulin: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.39' not found (required by cargo-tarpaulin)

I resolved the issue by avoiding binstall. Instead of running: 

I resolved the issue by avoiding binstall. Instead of running

> cargo binstall cargo-tarpaulin

I used the following command:

> cargo insatall cargo-tarpaulin

While this installation method is significantly slower, it successfully resolves the issue.

I suspect similar problems could arise with other Rust libraries when using binstall.

The non-experimentation result anti-pattern

Many of us have gone through the scenario of putting new software functionality into production and seeing tremendous improvement in business metrics. These results are often celebrated and attributed to this new achievement and this is used in presentations and corporate bonuses definitions.

But can we attribute causal relation between the new functionality and the new business metrics?

Before answering this question, however difficult it may be to abandon the emotive side of the functionality we have created, let us try to imagine some hypothesis that might have caused this improvement other than our change:

• An advertising campaign
• An unexpected viral
• The functionality created by another squad
• A concurrent product crashes
• It is a normal seasonal movement

These are just a few assumptions that could cause improvement in metrics. It is possible to mitigate each of them through some data analysis. But can you anticipate all other possible causal assumptions to be able to rule them out? Difficultly.

Since you can not assign causality between the two events, the most you can say is that there is a correlation between them. And yet it may be a spurious correlation. Some examples of this kind of correlations can be seen on this site here. It speaks, for example, about the strong correlation between the number of Nicolas Cage films per year and the number of drownings in the United States per year.

So the answer to the initial question is no! It is not correct to attribute the improvement of the business metric just because of its new functionality. If you want to check whether a new feature causes a change in some metric, an AB experiment is required.

In an AB experiment we have a variant (or alternative) called control (usually A) and a variant containing the modification to be validated (B). A random sample of users will receive the version of their product with variant A, and another random sample of users will see variant B. Everything in the two variants must be the same, except the modification made in variant B. With this you can control the environment and all those possible causes we reported above and more the others hypothesis we could not predict before.

At the end of the experiment we ate going to be able to attribute the causality of the new functionality to the incremental metric because it was the only thing different in the whole environment. Taking into account of course all statistical variance due to the use of a sample, which can lead us to possible false positives in a minimal percentage of cases.

AB experiments are the most accepted technique currently for you to determine the causality of a change in your product. They are not a new thing and are widely used as a part of the scientific method.

However there is currently a large discussion in the statistical area regarding causal inferences, which would be models for inferring cause from one event to another without necessarily an experiment controlling all variables. This discussion became even stronger after the publication of The Book of Why.

However, if you do not master the techniques of causal inference, it is best not to disclose cause and effect without being sure. I know it's very hard for us to leave our emotions aside for having participated in the development of that piece of software. But it's important to stay cool. And of course, always do AB experiments before putting something into production.

This post is part of a series about Experimentation Anti-Patterns, a talk that I presented recently.

Spark "first" function behavior on pandas dataframe

Spark first function is used to choose a value after aggregating some dataset value.

In python pandas we don't have this behavior as default after aggregating some dataframe, but we can do it easily we a few lines of code.

Since I could not find this solution on Stack Overflow. But first, let's see what happen with a column with string type when we do not use a function like firtst:

import pandas as pd
df = pd.DataFrame.from_records([
       dict(k=1, i=10, t="a"),
       dict(k=1, i=20, t="b"),
       dict(k=1, i=20, t="c"),
])
df.groupby("k", as_index=False).sum()

If we run the code bellow, we get this result:

  k i
0 1 50

You can see that the column t was removed since you cannot sum it.

Now, let's add the first aggregation function to this column:

first = lambda a: a.values[0] if len(a) > 0 else None
df.groupby("k", as_index=False).agg({'i': sum, 't': first})

If we run the code bellow, we get this result:

  k i  t
0 1 50 a

Solved

"Learning to ranking" with xCLiMF python implementation

At Globo.com we try to optimize our personalized recommendation providing better content in a ranked list. We have a hybrid system that can automatically essemble collaborative filtering, content based and non personalized algorithms.

But none of our implemented algorithms are intented to rank. Our best collaborative filtering algorithm, based on the paper Collaborative Filtering for Implicit Feedback Datasets, doesn't optimizes a ranking metric like MRR or NDCG.

Even so we have good results of those algorithms choosing the best hyper parameters focusing in MRR, we thought we could get better results using algorithms focusing in optimizing a ranking metric: Our first hypothesis is xCLiMF.

xCLiMF is a latent factor collaborative filtering variant wich optimizes a lower bound of the smoothed reciprocal rank of "relevant" items in ranked recommendation list. It is a evolution o CLiMF algorithm that can handle using multiple levels of relevance data instead of using only unary user preferences.

Implementation references

Since xCLiMF is very similar to CLiMF, I implemented xCLiMF based on this python CLiMF repository ( https://github.com/gamboviol/climf ). During the process I found an error and submitted a pull request ( https://github.com/gamboviol/climf/pull/2 ).

I have also found a C++ xCLiMF implementantion by other brazilian guy ( https://github.com/gpoesia/xclimf ). But this solution have two problems: The first one is a bug reported here ( https://github.com/gpoesia/xclimf/issues/1 ) and the second one is related to performance, since it updates user and item vectors separately as two loops of other two chained loops.

My implementation

So my implementation also have some differences from original papers:

• In the paper they don't mention any kind of previously normalization, but it's always a good practice to do it for machine learning algorithms input data. So I normalized dividing all rating by the maximum rating in the dataset. Using original ratings lead to some math errors:
• Exponential function of large values causes math range error;
• Exponential function of very large negative numbers get 0, and lead to some division by zero error; 
• Log function of negative numbers, caused by large exponentials, causes math domain error

• The paper experimentation setup uses N random ratings by user in training and in the cross validation data. Even so I have implemented this way, I have put an option to select 2N top ratings by user, randomly distributed in training and cross validation. For me, this last protocol reflected a more realistic kind of problem that xCLiMF is going to solve.
• In the original paper they described the algorithm as two chained loops. Firstly I implemented as described in the paper. But them I updated the code to use a more vectorized solution, having a 54% improvement in the performance. I maintained the chained loop solution for comparison purpose ( https://github.com/timotta/xclimf/blob/master/tests/looping_update.py )
• For the objective function, the paper describes that it's not necessary to divide by the number of users, because it is only used for validating if the gradient ascending is still increasing after many interactions. But I made this division for interpretability purposes.

Results

Using the experimentation protocol of N random ratings by user in training and in the cross validation, applied to the MovieLens 1M dataset, I got MRR variying from 0.08 to 0.09 at the 50 iteration using the hyper-parameters described in the paper.

Using the top K experimentation protocol in the same dataset I got MRR varying from 0.2 to 0.26. I tried those two experimentation protocol with the Alternative Least Squares implemented at Spark and the maximum MRR I got was 0.01.

The sequence of experiments, and how to run can be seen at https://github.com/timotta/xclimf

Conclusions and future

Now we have a working xCLiMF implementation in python, but it is not feasible to train it with Globo.com dataset. Running it using MovieLens 1M dataset that have 6k users took 4 minutes. With MovieLens 20M dataset that have 130k users took more than one hour. Globo.com datasets are much bigger than that, if we count only the logged users, our dataset of users preferences on shows has more than 4 millions users, it would take more than 40 hours to run.

That's why my next step is implementing this same algorithm on Spark. Since the updating user by user step is independent, it is possible to scale it running in parallel on our hadoop cluster.

I appreciate any help, correction, suggestion or criticism

References

CLiMF: Learning to Maximize Reciprocal Rank with Collaborative Less-is-More Filtering
Yue Shi, Martha Larson, Alexandros Karatzoglou, Nuria Oliver, Linas Baltrunas, Alan Hanjalic
ACM RecSys 2012

xCLiMF: Optimizing Expected Reciprocal Rank for Data with Multiple Levels of Relevance
Yue Shia, Alexandros Karatzogloub, Linas Baltrunasb, Martha Larsona, Alan Hanjalica
ACM RecSys 2013

Collaborative Filtering for Implicit Feedback Datasets
Yifan Hu, Yehuda Koren, Chris Volinsky
IEEE, 2008