App Mining Rewards Is Totally Broken


#15

I don’t mean to hijack all of Prabhaav’s points with this but I need someone to unequivocally tell me that “critic” the DE voter is not “critic.id”. No one has said that yet. Because those votes were absolutely counted and it’s a pretty odd coincidence that a voter gets allocated votes, drops all the votes on one app at the same time, and just so happens to have a DE username that matches a Blockstack ID of the developer that benefited from the vote.


#16

Justin, right on the same page with you.

Here's what we know for a fact at this moment: The user "critic" on Democracy Earth is tied to an investor who has been fully KYC'd as a part of the token sale and is not the creator of the app that benefited from the votes (completely different person, name and all). The above being said, the user who signed in to Democracy Earth either logged in with critic.id (the beneficiary app creator) or changed their username to critic . After logging in they confirmed their email address as that of the KYC'd investor.

These are the facts we know right now and are investigating further as to how this could have happened.


#17

It has been taken into consideration: see the spreadsheet. You’ll notice that Hermes DE Theta went from -1 to +0.897


#18

The fact that TryMyUI users were given incorrect instructions and there were bugs related to Blockstack affected our PH scoring.

What is the resolution for that? Are you asking us to carry forward a negative score due to no mistake on our part?


#19

Unclear the PH score was due to bugs, but we can find out pretty quickly. As for TryMyUI, their team will be on the call tomorrow so we can review with them. I’ve got access to all 8 of your TryMyUI user testing videos for this run so I’ll double check to see how it affected anything…I imagine there’s way their team can help tweak instructions to make it more fair generally (every app got the same instructions this time). I’ll be ecstatic when app mining runs smoothly, and given this is a pilot run there’s always a fair solution after thoughtful dialectic. Going to continue this discussion on the call tomorrow (it’s being recorded and will be posted to the forum) with the community so they can take part live. Thanks for being patient


#20

I’m going to continue the conversation in the forum because A) I and others, I imagine, won’t be able to make the conference call, and B) it’s important that anyone be able to see this discussion.

The problem with thoughtful consideration of how to introduce the Gaia storage requirement is that there are already apps on board that don’t use it. So you’ll put yourself in the position of telling those apps to add storage after the fact or grandfather them in.

I struggle with this point a lot because part of the reason a handful of apps participated in the alpha run in October was to provide feedback to help this process going forward. I mentioned this back then. Others did as well. I eventually even wrote a forum post about it. Now we’re deep into apps making significant money without putting in the work to use Blockstack’s core feature. As much as decentralized identity is cool, it’s not Blockstack’s important feature.

I’ve asked a few times about the progress toward Milestone 2’s goal of 1million users. If that goal is fueled by bringing on a bunch of apps that add Blockstack auth, I do not want to be part of that.


#21

+100 for making Gaia a requirement for app rewards mining!

I did have issues similar to Stealthy in regards to my app not always showing up on DE. Documented here https://blockstack.slack.com/archives/C074L9QQG/p1547739250356100

Also agree with mobile apps being at a huge disadvantage with the current process. The token bug really did destoy login for mobile. On web I was able to push quick patches and was at a huge advantage.

+100 on the positive side for taking our suggestions and using trymyui with relevent demographics of the person reviewing. The “video proof” that the reviewers did a due diligent job is great…but what happens if a reviewer does a poor job and it’s documented on video?


#22

Everyone, thanks a lot for your feedback. It’s very helpful.

Let’s first set the context here that we’re at a very early/experimental stage of App Mining, hence the pilot phase. This is not the “real App Mining”, this is the pilot run that’d help us iron out the issues for a smooth run of the main program. Even the main program has checks in-place for a “slow ramp up”, precisely to iterate and make the process better, smoother.

Any program like this will have issues. Period.

Decentralized app ranking is an extremely hard problem. People will try to game it. Some reviewers will not do a good job. Some app developers will occasionally get scores that should’ve been higher or lower. However, we can learn, improve, and repeat and that is precisely what we’re doing here. When we’re done, with the help of all of you, we would’ve solved an extremely hard but important problem and App Mining should rank apps on their quality in a transparent and hard to game manner, which is the goal here.

With that said, RE Gaia, I don’t think anyone is grandfathering any app in. There has been no discussion at Blockstack PBC to focus on the auth protocol as a way to reach Milestone 2. Gaia and decentralized storage is an important part of Blockstack apps and will get integrated. It’s a question of when and not if.

In the long run, any bad or low-quality reviewers will get voted out. In the short run, we can address the current concerns RE any specific reviewer. Nothing in the list of things listed in this thread is fundamentally concerning to the future of App Mining: they’re low-hanging improvements that can easily be made.

(Edited above sentence to clarify concerns about things being fundamentally broken vs quick fixes/improvements that are easily possible.)

Blockstack PBC team already pushes for transparency and already performs checks and audits. We can double down on this and push for even more transparency and audits as/if needed. Again, easy things. Remember that the goal is to make App Mining more decentralized and automated, so less reliance on PBC as that rolls out.

We’re off to an exciting start, we’re directionally going in the right direction, we’re getting valuable feedback from the pilot phase, let’s continue to improve and iterate!


#23

I don’t understand App Mining problem(s)?

Do we need app mining to incentives developers?
Do we need app mining to get more users?
Do we need app mining to make big investors happy?
Do we need app mining to make big promotion companies happy?
Do we need app mining to make our friends happy? ( * I make my friends happy many times for growing my company)

What’s the most important problem app mining wanna solve?

P.S. Have you guy ever think about that people like me don’t vote twice for any project? And don’t have time to vote on two systems. --> how many real users will vote on three system? (I guess None)


#24

Muneeb, the voter issue is concerning. I am far from convinced that the app developer himself did not supply those votes. I think anyone who has taken the time to look at the information Prabhaav supplied would be far from convinced as well. You should be concerned about that because, good reason or not, it looks bad. I know you all are looking into it, but you, of all people in Blockstack, should be concerned about this and many of the other issues.

App Mining is a perpetual pilot unless Stacks becomes publicly available. So this is what’s real to all of us right now.

The truth is, I’m terrified of the time I go from first to last or close to last. I work my tail off, putting in close to 40 hours a week on Graphite AFTER my day job. I know what I’m doing is valuable, but the system seems so flawed that I’m just waiting for that shoe to drop and Graphite to suddenly rank behind apps that clearly aren’t as well put together or not widely adopted. Or just flat out worse.

So my point is, I need you to be concerned about the items on this list because if you’re not, it’s a red flag.


#25

I think we’ve addressed that issue at least three times in this thread already and confirmed that the vote is from a valid token holder. Do you recommend not counting votes of valid token holders?


#26

Please explain how telling people it’s valid when there is evidence to the contrary is addressing it?


#27

I stopped doing everything I was doing, cancelled everything on my calendar, and spent the next 5 hours getting details on every single issue. I’m not concerned that these are any fundamental flaws in App Mining that can’t be fixed. Please keep the context in mind of that line.

If I was not concerned about this thread I won’t be spending time here :slightly_smiling_face:


#28

The account is linked to a token holder who is verified and is not the same person as the app developer. Did he convince a token holder to vote for him? Maybe. We don’t know.

Token holders can vote however they want and we can’t do anything about it especially in the long run.

My question remains the same. Is the recommendation here to censor votes of valid token holders? If yes how do we decide who to censor and who decides who can be censored? It’s a slippery slope and we all know the answer: we cannot censor token holder votes.


#29

@muneeb, I’m glad you’re very optimistic about things and spent considerable time looking into our concerns.

I know you feel that the question around critic.id has been addressed, but unfortunately, it’s not quite clearly explained. I’m still stuck on @patrick’s last statement: “These are the facts we know right now and are investigating further as to how this could have happened” I don’t understand how an ID linked to the beneficiary app creator is being used by an acredited investor–I know there are scenarios that could be happening (i.e. a hack, the app creator is a delegate for the investor, the app creator is the investor), but I don’t believe that has been clarified. It’s a very bad look without more information.

When I look at the totality of this situation, I can see that our biggest problem is that we are a mobile deployment competing against a sea of desktop apps. You’ve spoken about our product and been very kind, helpful and supportive to us in our effort to create a fine decentralized messenger. Indeed we would like to see a solution to App Mining Rewards that is fair. In this review cycle, the results do not seem to meet that standard for us and it is our responsibilty to the community to point that out.

If you believe that Stealthy is the lowest quality communication App on Blockstack then I suppose you might not see much an issue with the results. However our experience thus far seems to indicate a different reality and the results are not just suffering a minor error, they are flat out agregious. It’s difficult to understand how the results have put in last place the only App available on the most common hardware that uses push notifications over an App that literally does not allow you to send messages because of thrown errors.

In the long view, I agree that things can be fixed, but I’m less optimistic than you because the short term result seems so far off the mark.


#30

Looks like the investor was OK with his vote being used this way. It’s the investor’s choice. We haven’t seen any evidence of a hack but can certainly try to explore more.

Let’s take a step back and consider what’s going on here. Right now Blockstack PBC checks to verify that token holders and only token holders can vote, soon this can be done in a decentralized way. Anyone can verify the votes, token holders can vote however they want, they’d just be Stacks addresses on the blockchain. We’ve double, triple confirmed that only valid token holders voted in this round. What more can be done? Are there recommendations for censoring this particular token holder?

The concern is “did someone cheat and voted when they couldn’t have”. The answer is no, only verified investor votes counted.

Let’s even consider the hypothetical example if some token holder was hacked in the long run. What are we going to do? Take a private key and return it to someone? Do a fork and roll back the blockchain? As far as the system is concerned, in the long-run, the token holder is whoever has the private key and the vote is for whoever the owner of the private key votes for.

RE app quality in ranking, that is the sole purpose of App Mining. That’s our main focus. If App Mining can’t surface top apps then it fails and there is no point in it. I have some specific feedback on Stealthy and the UX and happy to give it over a call.


#31

Imo the app mining process has been hopelessly flawed since the beginning. It became obvious after the very first test run.

A lot of focus was put into showing of how reputable the ranking algorithm is, as it was developed by ivy league game theorists.

But unfortunately that doesn’t matter. As with any kind of data aggregation BS in = BS out and currently the group with most skin in the game (unpaid app devs) get no say in what efforts happening in their ecosystem should get rewarded while reviews by people who don’t have any knowledge of our values or context of what we are trying to achieve get to choose what is valuable.

And the message we get is ‘this is all fine’.

I have to say this confuses me a lot. I do believe muneeb and the team genuinely want this to succeed and are taking what they consider to be reasonable action to improve things, but somehow I doubt that the long run will work in their favor here.


#32

We’re all working to figure this out. Remember, the App Mining program is only a few months old. It’s silly to expect all kinks to be worked out by the 3rd month.

Make your voice heard by adding constructive feedback here.


#33

@bodymindarts thanks for the input, we got game theorists to do the job of designing the game and going deep on it, if there are specific improvements to the game, please do post them as a Stacks Improvement Proposal. In fact, any changes that you or anyone else would like to see should be posted as a Stacks Improvement Proposal. The community should discuss / debate and then pick the things that work and/or have more support.

In the extreme case if App Mining is totally broken and app developers are not being heard, they’d end up going somewhere else (even to a fork of Blockstack where App Mining works differently). Decentralized systems and open-source code has this beauty that in the end it’s community owned, it lives wherever the community lives.

I’d say all of you should feel like owners of this. Blockstack PBC, or me, or Patrick, or any single entity can’t do this alone. We need to act as a community. Disagree, debate, iterate, and improve.


#36

Maybe make some sort of script that analyzes how efficiently it runs Like the lease amount of errors in the code the more money you make or the least amount of resources you’re using