AppMining Scoring - A thoughtful proposal for how applications should be scored

#1

Hey Blockstack Community,

I have been thinking really hard as to what would could be a viable means to ending the confusion of how applications should be scored. With the most recent Zoom Meeting taking place, and Muneeb stating it may be better to end AppMining for a while until things are ironed out; I do believe we should all take some time to create some proposals to help the Blockstack Team!

The current AppMining list is full of applications that either carry no value or do not need a decentralized application to function. Some developers are pushing out multiple applications to raise their chances of scoring on the AppMining charts, when they could be grouped into one singular platform and offered once on the AppMining list. Some applications have been paid out too, when they are clear duplicates of open-source code posted onto Github and utilized to make a profit maliciously from the AppMining Program.

This being said, I have proposed an idea as to how AppMining can be properly developed on a number of key scoring methods that are both viable, straight-forward, and harder to manipulate. This is a work in progress and I would love input on the weight that each category should have on the overall Application Score.

  1. Awario (Mentions, Activity, Post Quality, and Backlinks) (20%)

My idea for Awario as that is should take into account several of the top social networks like Instagram, Facebook, Twitter, LinkedIn, and Reddit. Focusing less on Likes/Retweets, and focusing more on profiles that properly mention an application in their own posts. While Likes/Retweets should be taken into account, they should carry less weight on this score because they are easily manipulated by bounties and airdrops. By order of largest to smallest weight on Awario scoring, it should be:

  1. Mentions (How many profiles are sharing the account name? I.E. @MuseProtocol being shared by another user on Twitter)
  2. Back links (How many back links link to this post?)
  3. Post Quality (Average Likes/Shares it receives on it’s posts)
  4. Activity (How often are they engaging with their community?)

Each of the above items should weight into the overall Awario score to give a solid score of 1 out of 20. This will keep the scoring slice of Awario to a solid 20% of the score. Overall score should have a weight of 20 points from this subject.

  1. TryMyUI (User Interface, User Experience, and Design) (20%)

The User Experience is of utmost important to the growth and psychological grip that an application has on it’s user base. Too complex and an application will see it’s own learning curve defeat it’s potential at any chance of success. Too simple, and people won’t be able to see the true potential something has in being a key part of their life. This being said, the User Interface, User Experience, and Design should be heavily weighted in the scoring. This is what will gives Blockstack it’s massive support from the community is that their applications all have an extremely original look, feel, and use to them.

Every month, a panel of 100 randomly chosen paid Blockstack ID’s should receive an email or message stating they have been chosen by TryMyUI to review a list of applications honestly and to the best of their abilities. This will keep the ratings from being manipulated and will decentralize the overall approach to who says what about an application. There should be 20 questions on this review sheet for each reviewer, and a simple Yes (1) or No (1) answer to each. All of these review sheets should be received and then for each question that receives a majority yes or majority no, they should receive 1 point towards the 20 possible points an application can receive from TryMyUI. A 50/50 vote on a question should receive an automatic No (0) on that point because an undecided should be weighted more as a No due to the fact that a Yes should be easier to decide. Overall score should have a weight of 20 points in this subject.

  1. Blockstack Voting (Paid Blockstack ID’s should have a say in the AppMining) (15%)

One part of the current means of scoring utilized by AppMining that makes absolutely no sense, is that Paid ID’s have no say in how apps are scored in the ecosystem. This being said, this needs to change immediately. I propose that every month during the scoring period, all paid Blockstack ID’s should be able to login to the AppMining website, and submit a (+) or (-) vote on an application to help it’s score for that month’s AppMining. Keeping this to paid ID’s ensures that manipulation can be kept at bay as it would not benefit anyone to pay enormous amounts of Bitcoin to create more paid ID’s just to submit a vote. This will keep the scoring organic and also give Blockstackers a say in what applications get scored in their ecosystem. Overall score should have a weight of 15 points in this subject.

  1. YourNetInternet (Blockstack Authentication and Gaia Storage) (15%)

Since the application built is based on Blockstack, it should obviously utilize some form of Blockstack Authentication and implementation of Gaia Storage. Currently, the scoring for this is relatively well organized so far, and should remain the same. Applications should be scored on whether Blockstack is their primary use of authentication, only use of authentication, or if it is not utilized at all. Applications should be scored on if they clearly utilize Gaia Storage, if it is unknown of they use Gaia Storage, of if they do not use Gaia Storage at all.Overall score should have a weight of 15 points in this subject.

  1. Can’t Be Evil (Can’t Be Evil Sandbox Implementation and Open-Sourcing) (10%)

The implementation of the new CBE Sandbox is an amazing feature indeed. It should be utilized in scoring to ensure every application has this implemented so a user can see what is being requested from them that could be potential identifying or be used maliciously for intent of collecting user based analytics. This should be a mandatory scoring that lowers the potential score for each element of CBE that should not be present in a Blockstack decentralized application. This should make up 7 of the 10 possible points for this subject. The remaining 3 points should be based on open-source code. 0 points if there is no open source code, 1 point if they have some code present, and 2 points for most of the code present, and 3 points for all of the developments source code present. Overall score should have a weight of 10 points in this subject.

  1. Activo (Daily, Weekly, and Monthly User Activity Ratings) (10%)

This should be a record of counting how many active users are using applications on a daily, weekly, and monthly basis. With heavier scoring based on the amount of daily active and returning users an application has. This is important because we want to know which applications have usage in the ecosystem to begin to see what type of ecosystem is developing around the Blockstack community. With 10 possible points in this category, 5 points should be earned for daily active users returning, 3 points for weekly active users returning, and 2 points for monthly active users returning. This would be scored utilizing metrics compared to other applications in the ecosystem and averaged out to see where the proper score should be placed for each category of scoring for each application. Overall score should have a weight of 10 points in this subject.

  1. Blockstack Culture (Blockstack Culture Team Rated Scoring) (5%)

This would be a means for the official Blockstack team to take part in scoring applications. They would review if the application fits into the culture of what they believe Blockstack stands for. This would be less of a code review and far more of a product review as a whole. The team would review the applications, their business models, their goals as a company, and their current use case, and give it a rating between 1 and 5 making up 5 possible points to earn from this category. This will help keep majority of the scoring decentralized, but still give the Blockstack PBC team a solid way to give that extra push for a development to get up in the rankings if they believe it is a development that prides itself on following the Blockstack Development Culture. Overall score should have a weight of 5 points in this subject.

  1. Originality (Current Saturation of the Market for the Application) (5%)

This would also be a means for the official Blockstack Team to take part in scoring applications. They would review the current applications in the ecosystem and see if the market they are attacking in the ecosystem is original or if it already saturated to a point that it could NEVER earn these 5 points. This will help keep ideas fresh, original, and new in the ecosystem to ensure people do not attempt to duplicate ideas over and over again. A score of 5 points will be available in this category, with a 0 provided if the market already has 10+ developments in it. 1 point if it has 8+ developments, 2 points if it has 6+ developments, 3 points if it has 4+ developments, 4 points if it has 2+ developments, and a solid 5 points for being the first in the market to attack that industry. Overall score should have a weight of 5 points in this subject.

The above means of scoring equals out to a possible 100 points to earn from scoring. Which can equate to a score of 100%. This could be a great way to properly score and organize each application for the years to come. It is obviously not perfect yet, but with some community input, I am sure we can iron out the best ways to score these applications. As Muneeb said, although they manage AppMining, we own it! We should take pride in devising plans to have it work as efficiently as possible so that Blockstack, and ourselves, are not being robbed by applications that clearly do not deserve any rewards at all.

I would love to hear any and all community input on this!

Let’s start the year off right with solving the biggest issue of all to date, how our applications should be scored!

2 Likes
#2

Awesome please join us on discord to discuss as well.

Just some quick questions and comments and my math probably wont add up.:
#4 “YourNetInternet” - what is this? just auth and storage use? If so, I’d weight this much much lower. That is the lowest possible bar otherwise, why even call it a blockstack app? 5 %.
#5 10% - This is the only true test of Cant Be Evil we have at this point, should be weighted much higher. This is arguably the most important part. 40%
#6 Activo - we don’t have a good way count users in a decentralized way at this time. Muneeb mentioned this as one of the hard things they’re working on. 0%. Absolutely should be the key metric once measuring is solved.
#8 Originality - i like the idea, but its too easy for me to quickly add 10 of any type of app here and attack other app miners. 0%

#3

Thanks so much for this proposal @anthony.id!

I really like the Blockstack Culture component. This is something that would be great to have. One thing that comes to mind though is the massive amount of time it would take to do this. This month we had over 300 apps. If someone spent 5 minutes looking at each app with no breaks, that would be over 25 hours or over 3 work days. To even briefly review “the applications, their business models, their goals as a company, and their current use case,” you’d probably need to spend a lot more than 5 minutes. Probably at least 30 minutes to an hour per app. Since it’s such a subjective, arbitrary measure, we’d also probably want to have panel of people each give a rating and average out the scores. It quickly becomes unscalable.

What we’re trying to do with both the Can’t Be Evil Sandbox and our upcoming browser and implement a lot of these Blockstack cultural values in code so that if an app runs in our browser you know that you’re not being tracked unless you gave permission, your data is getting stored in your gaia hub, etc.

Perhaps this combined with a manual Blockstack Cultural Values review as you suggested of the top 5 or 10 apps ranked on all other metrics would make it more scalable.

Thanks for getting us thinking!!

#4

First of all thanks for submitting this proposal, this is useful. Just going to comment on the below to determine ways to prevent its gaming:

Does this not create an issue where folks enter a race to buy up the requisite number of paid IDs to keep their score high, thus sybil-attacking the ranking mechanism? Getting provably unique identities, and additionally preventing vote buying is incredibly difficult to solve for.