App Mining Improvement Proposal: Pause Democracy Earth as Investor Community App Reviewer

#1

During the App Mining pilot, Democracy Earth was invited to become our Investor Community App Reviewer. The intention was that they would help rank apps based on the sentiment of verified Stacks token holders. We now have several months worth of ranking data allowing us share our learnings and suggest a path forward, concluding with a proposal to remove the Investor Community App Reviewer until we have data-backed solutions to some of the existing problems around voter incentives and engagement.

Democracy Earth Overview

Democracy Earth’s platform is open to verified Stacks token holders. Investors who opted in received on-boarding instructions to the Democracy Earth platform with their Blockstack ID, with guidance to evaluate all of the App Mining apps, and distribute all of their 1000 lifetime votes.

Token holders were prompted to cast their votes by voting an application either up (signifying approval) or down (disapproval). Token holders could have changed their votes at any time and had instructions to reallocate or maintain votes given the previous month on one application to different ones on a monthly basis, and received email reminders at the start and midway of each months ranking period.

Token holders could have abstained from voting as well. App Miners were able to participate in the Debate section of each app and respond to questions or feedback from the Voters. The full overview is here.

Investor engagement challenges

To start the program, we emailed 600/800 current investors (most of them twice) with a personal email to invite them to the program. We also manually on-boarded token holders over video calls to address questions and sell the value of voting. Some stats:

  • We converted 176 token holders agree to become voters via email.

  • When it came to actually voting on the platform, only 57/176 actually voted.

The breakdown by month:

  • December: 20 token holders voted

  • January: 30 token holders voted

  • February: 42 token holders voted

  • March: 49 token holders voted

Motivating additional investors to vote has felt like a slow – albeit tractable – problem. However, motivating those investors who voted once to reevaluate apps on a monthly basis has been even more challenging.

We’ve observed a clear pattern of investors setting votes once then never re-casting them. This has clear downsides for new apps. For example, in March we saw 18 token holders revote vs. the 57 total lifetime voters, or about 31%. By extension, we’re also seeing this pattern play out in the average payout for apps depending on what month they joined App Mining:

For App Mining to be successful, new apps cannot be penalized simply for joining after other apps. And as that relates to token holder voting, we need a reliable incentive to encourage reevaluation of apps on a regular basis. This challenge is not altogether unexpected since token holders aren’t paid for voting, and assessing 60+ apps requires serious time.

Addressing the problems

PBC has conferred with Democracy Earth and considered a number of solutions:

Improving the Democracy Earth interface: There may be product design improvements which will reduce the burden to evaluate all apps, and potentially revote, more frequently.

This option seems experimental and might take months to prove its effectiveness.

Resetting or decaying the votes: We considered a clean state each month and also a decay function to help nullify our challenge with reevaluation.

This change would probably help but would not address the fundamental lack of incentives to revote.

Switching the voting population: There have been public requests to switch from Token holders to another voting cohort, such as App Miners, or Blockstack forum members.

  • Longterm, we still believe token holders are the right voting cohort—they own part of the Blockstack ecosystem and are financially incentivized to help the ecosystem become successful, which means supporting high-quality apps.

  • Switching away from token holders may, or may not, solve the voter engagement challenge. We would prefer to get the incentives right vs. hope a new audience will care more.

  • App Miners have conflict of interest that we predict will produce a poor ratio of signal to noise. The current token holder voting already has this problem, and we feel it is guaranteed to increase if App Miners vote on apps.

  • We feel forum membership is an abstract cohort and an inferior voting cohort to both App Miners and token holders.

PBC’s current conclusion

We may be wrong on some of the assertions or challenges listed above, but one thing is certain—it will take months of research and development from a dedicated team to test these ideas. The PBC team doesn’t want to gamble with your payouts while we wait for those results.

The PBC team is small. The Democracy Earth team is small. Our priorities lie in quickly improving, and reducing friction in, the App Mining ranking system.

We still believe that ranking apps based on evaluation by Stacks token holders is, in principle, a high-potential app reviewer. However, given:

  • The urgency of this problem

  • The current size and breadth of our token holding population

  • And the lack of proper incentives for voting;

The PBC team feels the Investor Community App Reviewer is not delivering the value it could and should deliver and that a resolution will take months.

Democracy earth has a ton of exciting projects and work ahead, including partnerships with local and state governments, and are currently testing out quadratic voting. We are extremely close with their team and it is no fault of theirs that we feel this is the best path forward for the time being. We will continue to support them.

Next steps

Our recommendation is to pause Democracy Earth, and the Investor Community App Reviewer, from the App Mining ranking until we can address the challenges described above. This change would go into effect May 1.

Feedback and thoughts welcome.

4 Likes
#2

I see alot of really good apps like Misthos, Stealthy, Cryptocracy and Afari get beat by very simple photo or blogging apps. So there is definitely something wrong and I don’t think it’s isolated to Democracy Earth. Unfortunately I think this has caused a lot of really good and smart developers to disappear from the community.

A community member vote is the most obvious solution, which i think the token holders and investors should also be a part of!

2 Likes
#3

We should have a protocol which can change/remove/add any App Mining protocol. That said we need a governance protocol.

Explain: We cannot blame anyone if the whole community support a certain rules. But we can blame Blockstack PBC for what it is now because they’re the one in charge and people will blame other to support or not support Blockstack PBC and stuffs. Of course, if everything go well >>> Blockstack PBC is pretty much like Elon Musk = the best.

#4

Thank you for laying out the numbers on engagement Gina. They definitely reinforce what we see anecdotally.

+1 for pausing DE votes for app mining.

3 Likes
#5

It was naive to think that any millionaire has anytime to download and play with even one app, leave alone 50+ apps. It is still naive to think retail investors (outside of active dev community and forum members) will taking voting seriously.

PBC as a centralized entity should have acted really fast if a mistake was identified earlier in the process (several community members identified and wrote about it) rather than waiting for 5 months to collect enough data to support your argument. After all every mistake is costing real devs who were early adopters of the platform. There are still several others bugs in the system. I was just looking at April payouts, $0.03 payout to support development of a particular app is a practical joke! If the app is not worthy enough it shouldn’t be in the program, if it’s worthy enough it should be given fair reward to keep the founders going with the difficult task of development and on-boarding users.

Programs like this are about seeding a healthy eco-system while you folks are keeping busy in designing a maze (that could conform to your game theory) for devs that can actually easily find something else to do with their lives.

1 Like
#6

Agree! Almost nobody can try 50 apps a month. I doubt Muneeb and Ryan try 50 apps a month.

I know the fact that app mining is not only for dapp developer. There are many entities involve here. From the business side, it’s normal to make some of the board members happy, some big investors happy, some friends happy or some employees happy >>> Right now, I’m doing exactly what Blockstack does = sharing the cakes. But my business is traditional business.

Blockstack PBC want both: some entities and majority (dapp devs, users and small investors). But there’s a trade off here. If you make an entity happy - you got them.

Yes, Blockstack PBC make some entities happy - not the others >>> of course, mathematically they have to loss dapp devs, users and small investors.

You should look at the money payout and you will see clearly what they value most.


And this is how much stacks they hold

So you have their bonus = app mining reward for top dapp in one year. I’m not even talk about option reward. From what I see, clearly the top dapp (Graphite) contribute less than 1 person work.

I’m not even ask how much Blockstack PBC pay for expert reviewers.

Agree! Anybody can do better.

My advice for Blockstack PBC is, the probability of success is not directly in your hand though >>> It’s from majority and community. Choose wisely when you still have choose option.

And my advice for all people is, value your time!

Peace!

1 Like
closed #7