Agnosticis in the Smart Contracts Industry

Agnostic

agnostic

Although we at TNW love new tech, we’re also a bunch of cynics that view it with a certain scepticism (I mean we’d be lousy journalists if we didn’t). We, like you, know what practices we consider bad — such as irresponsible handling of personal data — but it’s harder to pinpoint what’s . good That’s why I asked Torben Friehe, co-founder of the , a European forum for piloting technological advances, what exactly is _good_technology? Good Technology Collective “I don’t know.” Well, that was somewhat underwhelming. (But worry not, I only chose this as the first quote for dramatic effect and to remind you that nothing in life is easy — the truth is that this article is actually worth reading.) The GTC was founded to bring together leading figures in tech from different disciplines — e.g. science, politics, journalism, etc. — to offer counsel on the societal impact of new tech, such as AI, data surveillance, and IOT. So how come Friehe can’t simply say what good technology is? “The thing is that the GTC is founded on seeking the answer to that question. We don’t just want perspectives we agree with, we want to foster real discussion on which direction we want to take tech. The answer could be that technology is bad in principle and we should go back to our caves, I don’t know,” says Friehe and adds laughing “if someone convinces me that’s the right approach, I might live in a cave.” all But jokes aside, Friehe has strong opinions on what good tech is, even though he acknowledges that he alone doesn’t hold all the answers. Intentions, purpose, and accountability (you know, all the fun stuff) “Whether technology is good or bad depends on the people that create it — and I think that’s one of the things that’s neglected. Technology doesn’t appear out of thin air, there are people behind that create it. These people are the ones deciding whether the tech has positive or negative qualities,” says Friehe and emphasizes that engineers also need to become more deeply involved, and question the purpose of the tech they contribute to. Friehe calls for increased accountability among engineers, not just the ‘decision makers’ that form the original intentions or the purpose of a product — whether it’s military tech or data products. “There are a lot of people that are working on technology that don’t even know about the intention behind the product they’re developing. And that’s particularly the case with, and always has been, with engineers.” In Friehe’s opinion, engineers working on frontier technologies are too free to approach their task with almost a child-like zealousness. They can focus solely on the engineering challenges and the joys of solving them, without questioning or even thinking about the end-use of the tech. “The intention of the product is rarely influenced by the engineer. The engineer receives directions and executes them, but we need to change that,” says Friehe. The entire development chain of tech needs to be more interactive and influential and the first step towards that is having engineers have more say. Technology with a purpose — and built with restraint Tech is developing so fast that there are hardly any limits to what we can do, that’s why need to create the restraints — because computing power doesn’t anymore. In Friehe’s opinion, the best way to do that is to educate engineers in ethical related issues and give them more power in forming the purpose of the product. we “The input that engineers give on designing certain solutions influences the design, for example the extent of data collection. Engineers, at least at my company, have a lot of influence on what data we should or shouldn’t record,” says Friehe. The reason for that is that the purpose of the product will ultimately guide where we employ restraint in tech. “It might be a stupid analogy but if you’d collect all the paper documents that you get at home — and literally not throw away anything — your apartment would be flooded with letters.” Friehe feels that engineers of data products need to think about data collection in a similar way. We have the capacity to fill our apartment with letters, but it’s not a smart thing to do, especially if it can be harmful to other people. Although Friehe doesn’t mention it explicitly, I can’t but help but to think about purpose as a reaction to the mentality of advancing tech for tech’s sake. Of course it can be argued that uninhabited innovation has given humanity great gifts, but that’s an attitude that better suited in academia. The reason is that businesses in the private sector (where most technological innovation happens nowadays) always have an underlying and inherent purpose. Whether it’s creating wealth for shareholders or ‘disrupting’ a market, the purpose is there — so it’s important that everyone that participates in making it a reality is a aware of it. So for me, technology might frankly be as simple as technology for the sake of people. But I bet that corporate big wigs could find a way to twist that into something ugly and, for example, argue that people need tech overlords to monetize their personal data to provide them with a platform to argue with their racist uncles. good However, Friehe and the members of GTC are bigger people than me and recognize the need to include all perspectives to reach a meaningful conclusion — because no one group can dictate tech. “There is no single authority that can determine which technology is good or bad,” says Friehe. “And to be clear, it should never be the GTC’s role to determine whether particular tech is good or bad. It should be a forum to discuss what individuals and organizations is good or bad.” think The first step in answering what’s good tech “One of the objectives of the GTC is identifying the threats to good technology. Of course I have an opinion there, but I need to find out whether it’s valid or not. The problem is that people don’t talk about these things, maybe sometimes at pubs — but unfortunately, most influential tech people don’t spend that much time in pubs.” That’s why Friehe and his co-founder Yann Leretaille formed the GTC along with its , which includes influential people in different areas of tech, such as: Expert Council Luciano Floridi, professor of ethics of information and the head of the Oxford Internet Institute Anke Domscheit-Berg, member of Germany’s parliament with the Die Linke party and activist with ties to WikiLeaks Annie Machon, former MI5 counterintelligence officer, data activist and whistleblower Rachel Coldicutt, CEO of Doteveryone, a London-based think tank fighting for a fairer Internet Gerd Leonhard, a top-rated futurist, author and strategic advisor Matt McMullen, founder of RealDoll and Realbotix, the leading AI companionship startup Ida Tin, founder of the internationally acclaimed female health app, Clue Friehe feels that now, for the first time, we actually have the capacity as a society to discuss whether we want new technology, whether we need it, how we want it to be, and what we should do with it. He genuinely believes that if you have a forum to display arguments on good tech — and if you get intelligent people to listen — then people will eventually make up their minds. The GTC will officially open its doors in Berlin on December 15 and you can attend by registering . But despite it only being officially launched now, TNW has been monitoring its inception for the last few months. We’ve even published a few articles by some of their experts — e.g. on , , , and — and it looks like the collective is off to a good start. here algorithms spies and digital privacy problem of disruption AI sex dolls All the original costs of GTC were funded by member donations and in 2018, the collective will operate as an e.V., a nonprofit according to German law, funded through grants and non-corporate donations. But the GTC still needs to be careful with its co-operation with big tech if it wants to make a meaningful contribution to answering what is good tech. Friehe acknowledges that it isn’t just up to him, but the intention is to keep GTC as independent as possible and make sure that future partners cannot influence the agenda of the collective too much. Big tech, just like any other participants, have a right to their opinion on good tech, but it needs to clear that it’s their stance, and not masking it with GTC. Just like you, I’d hoped that this article could’ve given an definite answer to what exactly constitutes as good technology, but it’s too big of a question for now. One thing though is for sure, we’ll never know what good tech is unless we keep asking the question. This story was written by Már Másson Maack , and was originally published on The Next Web. To read the weekly trending TNW story on Hacker Noon, visit Hackernoon.com/tnw .

Technology is Agnostic. The People Who Make it Aren’t.

Companies are now turning to data as one of the most important assets in their businesses, and data engineers are in the midst of managing and improving this asset and its effectiveness. In addition, the integration of data engineering with the use of cloud computing offers scalability, accessibility, and reduced expenses. However, there are security challenges that come with this integration that need to be solved. It is essential to know the rates and causes of cloud security incidents in order to identify the vulnerabilities of the cloud systems. The cost of security incidents in businesses has shockingly increased to 61% in 2024 from the previous year’s 24%. As a data engineer, you are in a position to manage the flow of data in a society that is constantly in evolution. This responsibility is very important in today’s organizations, and if not given due attention, it can affect its overall performance. With the increasing use of cloud technology, it has led to an increase in the frequency of migration processes. It is now evident that cloud security is no more a concern only within the IT department. It has become a crucial factor in the overall business process and productivity. Why is this so important? It is because any wrongdoing or negligence in the system may lead to adverse consequences such as system failures that interrupt business flow, bad image, and breach of data privacy. However, a problem I often see businesses struggle with is that they have their cloud platforms varied and the security tools they use are specific to each ecosystem including the AWS, Azure, or Google Cloud Services. It is possible to become an expert in one system, and this may not allow you to easily transfer your skills to another system. That said, there is a solution to this challenge in the form of an approach that can work on any platform. This means that one can find a number of security principles and practices that are not aligned with any particular vendor to develop a solution that can scale across different clouds. In this piece, I’ll list down the basic principles that a data engineer should know about cloud security and share some insights on the current developments in the field. Key Fundamentals of Cloud Security for Data Engineers Data Encryption Encryption is one of the most effective methods of ensuring that information cannot be obtained by unauthorized persons with ease. When data is stored (At Rest) on the cloud, generally, there are commonly used encryption standards such as Advanced Encryption Standards (AES) with a 256-bit key by the leading cloud service providers. For instance, in AWS, users can simply secure their S3 storage buckets by configuring the encryption settings. To secure data while it’s being transferred from one point to another point (In Transit) on the network, infrastructure upholds TLS (Transport Layer Security) which encrypts communication between services. Access Control and Identity Management The allocation of access control helps in preventing the occurrence of attacks by implementing the notion of least privilege to users and applications and granting them only the required access to perform their tasks on the cloud platforms such as GCP IAM, AWS IAM, or Azure Active Directory where administrators use tools for permission and identity management. The provided example showcases an IAM policy in AWS designed specifically for limiting access to S3 buckets. Network Security The basic infrastructure of cloud security comprises a network system that has features such as Virtual Private Cloud (VPC), network access control lists (NACLs), and firewalls that regulate incoming and outgoing traffic. For instance, in Azure, we can create a Network Security Group (NSG) to filter traffic as shown below. Secure Data Storage and Processing Cloud data must be kept safe while still being easily accessible for processing purposes. Technologies like Bucket Lock in Google Cloud Storage and AWS S3 Object Lock are designed with features that enable immutable data storage in the native environment. Monitoring and Auditing This makes it possible to detect threats in cloud environments since user actions and system events are examined as well as AWS CloudTrail or Azure Monitor logs for any anomalies. Here is a demonstration of how to activate CloudTrail for an AWS account. Compliance and Data Governance Compliance frameworks like GDPR or HIPPA and standards such as SOC 2 require data engineers to incorporate security measures to protect sensitive data adequately. In order to remain compliant with the regulations in place and maintain security standards for information classification and tagging of data, these processes should be conducted by data engineers in collaboration with relevant security teams. As an example of this practice in action, Azure Purview has the capability to automatically categorize data based on the policies that have been applied. Securing Data Pipelines Data pipelines can also be susceptible to man-in-the-middle attacks or incorrect configuration if misshapen by individuals in the middle of a communication channel or if settings are misapplied or misaligned. When choosing among the available candidates for your data pipeline, it is recommended to use either AWS Glue or Apache Airflow as they both provide ways of connecting to your data in a secure manner. For example, the following is a sample setup guide on Apache Airflow which demonstrates the application of an encryption process for enhancing security. Emerging Cloud Security Trends Dealing With an Influx of Data The issue of data volume in cloud environments is a double-edged sword as it creates as many benefits as it does challenges. It’s a well-known fact that the volume of data alone can be quite daunting for tools to manage. The real difficulty, however, does not reside in the capture of data but in the analysis of it - identifying correlations in order to recognize trends and anomalies as well as predicting security threats. Nevertheless, data engineers can employ technologies that go beyond the standard ones and are enhanced by analytical power, Artificial Intelligence, and machine learning. These techniques have the potential to take the currently unstructured data streams and turn them into valuable insights that enhance the overall security posture within the continuously evolving digital environments. Zero Trust Architecture The traditional security model which is based on the concept of perimeter defense is slowly being phased out and giving way to the new zero trust model. The basic tenet of zero trust is based on the premise that no component whether it is a user or a device should be trusted automatically. A major advantage of this methodology is that it focuses on the validation of the user identity, the device, and the access rights that are to be granted. Thus, employing tools that are compatible with various platforms enables data engineers and security personnel to adopt this very effective protective measure in different cloud environments. The Development of Cloud Security Products In a world where cloud services are commonly used and architecture is rather complex, securing systems throughout the systems is not an easy task. This has led to the development of cloud security management tools, which ideally should be able to integrate with other tools to enhance the work of the security teams. The main aim is to enforce the implementation of security policies and gain control and visibility of the different cloud stacks. This should ensure that security measures are put in place irrespective of the technology that underlies it. In Summary Security in the changing world of the cloud is like going on an adventure, and that is a good thing. It is important to deal with security concerns and vulnerabilities when they surface - this puts you in the lead. Zero trust frameworks offer important validations when it comes to handling security issues while the unified security architecture guarantees that policies are well enforced and there is a proper view. But the architectural design of cloud environments is still in the process of evolving, and they need to be adaptable and proactive in order to enhance the shielding. Although all the strategies mentioned here and the ongoing efforts to match up with the advancements can be used to enhance the security of major cloud infrastructure, the threat cannot be entirely ruled out. Companies are now turning to data as one of the most important assets in their businesses, and data engineers are in the midst of managing and improving this asset and its effectiveness. In addition, the integration of data engineering with the use of cloud computing offers scalability, accessibility, and reduced expenses. However, there are security challenges that come with this integration that need to be solved. It is essential to know the rates and causes of cloud security incidents in order to identify the vulnerabilities of the cloud systems. The cost of security incidents in businesses has shockingly increased to 61% in 2024 from the previous year’s 24%. increased to 61% in 2024 increased to 61% in 2024 As a data engineer, you are in a position to manage the flow of data in a society that is constantly in evolution. This responsibility is very important in today’s organizations, and if not given due attention, it can affect its overall performance. With the increasing use of cloud technology , it has led to an increase in the frequency of migration processes. cloud technology cloud technology It is now evident that cloud security is no more a concern only within the IT department. It has become a crucial factor in the overall business process and productivity. Why is this so important? It is because any wrongdoing or negligence in the system may lead to adverse consequences such as system failures that interrupt business flow, bad image, and breach of data privacy. However, a problem I often see businesses struggle with is that they have their cloud platforms varied and the security tools they use are specific to each ecosystem including the AWS, Azure, or Google Cloud Services. It is possible to become an expert in one system, and this may not allow you to easily transfer your skills to another system. That said, there is a solution to this challenge in the form of an approach that can work on any platform. This means that one can find a number of security principles and practices that are not aligned with any particular vendor to develop a solution that can scale across different clouds. In this piece, I’ll list down the basic principles that a data engineer should know about cloud security and share some insights on the current developments in the field. Key Fundamentals of Cloud Security for Data Engineers Key Fundamentals of Cloud Security for Data Engineers Data Encryption Data Encryption Encryption is one of the most effective methods of ensuring that information cannot be obtained by unauthorized persons with ease. When data is stored (At Rest) on the cloud, generally, there are commonly used encryption standards such as Advanced Encryption Standards (AES) with a 256-bit key by the leading cloud service providers. For instance, in AWS, users can simply secure their S3 storage buckets by configuring the encryption settings. To secure data while it’s being transferred from one point to another point (In Transit) on the network, infrastructure upholds TLS (Transport Layer Security) which encrypts communication between services. Access Control and Identity Management Access Control and Identity Management The allocation of access control helps in preventing the occurrence of attacks by implementing the notion of least privilege to users and applications and granting them only the required access to perform their tasks on the cloud platforms such as GCP IAM, AWS IAM, or Azure Active Directory where administrators use tools for permission and identity management. The provided example showcases an IAM policy in AWS designed specifically for limiting access to S3 buckets. Network Security Network Security The basic infrastructure of cloud security comprises a network system that has features such as Virtual Private Cloud (VPC), network access control lists (NACLs), and firewalls that regulate incoming and outgoing traffic. For instance, in Azure, we can create a Network Security Group (NSG) to filter traffic as shown below. Secure Data Storage and Processing Secure Data Storage and Processing Cloud data must be kept safe while still being easily accessible for processing purposes. Technologies like Bucket Lock in Google Cloud Storage and AWS S3 Object Lock are designed with features that enable immutable data storage in the native environment. Monitoring and Auditing Monitoring and Auditing This makes it possible to detect threats in cloud environments since user actions and system events are examined as well as AWS CloudTrail or Azure Monitor logs for any anomalies. Here is a demonstration of how to activate CloudTrail for an AWS account. Compliance and Data Governance Compliance and Data Governance Compliance frameworks like GDPR or HIPPA and standards such as SOC 2 require data engineers to incorporate security measures to protect sensitive data adequately. In order to remain compliant with the regulations in place and maintain security standards for information classification and tagging of data, these processes should be conducted by data engineers in collaboration with relevant security teams. As an example of this practice in action, Azure Purview has the capability to automatically categorize data based on the policies that have been applied. Securing Data Pipelines Securing Data Pipelines Data pipelines can also be susceptible to man-in-the-middle attacks or incorrect configuration if misshapen by individuals in the middle of a communication channel or if settings are misapplied or misaligned. When choosing among the available candidates for your data pipeline, it is recommended to use either AWS Glue or Apache Airflow as they both provide ways of connecting to your data in a secure manner. For example, the following is a sample setup guide on Apache Airflow which demonstrates the application of an encryption process for enhancing security. Emerging Cloud Security Trends Emerging Cloud Security Trends Dealing With an Influx of Data Dealing With an Influx of Data The issue of data volume in cloud environments is a double-edged sword as it creates as many benefits as it does challenges. It’s a well-known fact that the volume of data alone can be quite daunting for tools to manage. The real difficulty, however, does not reside in the capture of data but in the analysis of it - identifying correlations in order to recognize trends and anomalies as well as predicting security threats. data volume data volume Nevertheless, data engineers can employ technologies that go beyond the standard ones and are enhanced by analytical power, Artificial Intelligence, and machine learning. These techniques have the potential to take the currently unstructured data streams and turn them into valuable insights that enhance the overall security posture within the continuously evolving digital environments. analytical power, Artificial Intelligence, and machine learning. analytical power, Artificial Intelligence, and machine learning. Zero Trust Architecture Zero Trust Architecture The traditional security model which is based on the concept of perimeter defense is slowly being phased out and giving way to the new zero trust model . The basic tenet of zero trust is based on the premise that no component whether it is a user or a device should be trusted automatically. zero trust model zero trust model A major advantage of this methodology is that it focuses on the validation of the user identity, the device, and the access rights that are to be granted. Thus, employing tools that are compatible with various platforms enables data engineers and security personnel to adopt this very effective protective measure in different cloud environments. The Development of Cloud Security Products The Development of Cloud Security Products In a world where cloud services are commonly used and architecture is rather complex, securing systems throughout the systems is not an easy task. This has led to the development of cloud security management tools , which ideally should be able to integrate with other tools to enhance the work of the security teams. The main aim is to enforce the implementation of security policies and gain control and visibility of the different cloud stacks. This should ensure that security measures are put in place irrespective of the technology that underlies it. cloud security management tools In Summary In Summary Security in the changing world of the cloud is like going on an adventure, and that is a good thing. It is important to deal with security concerns and vulnerabilities when they surface - this puts you in the lead. Zero trust frameworks offer important validations when it comes to handling security issues while the unified security architecture guarantees that policies are well enforced and there is a proper view. But the architectural design of cloud environments is still in the process of evolving, and they need to be adaptable and proactive in order to enhance the shielding. Although all the strategies mentioned here and the ongoing efforts to match up with the advancements can be used to enhance the security of major cloud infrastructure, the threat cannot be entirely ruled out.

A Platform-Agnostic Approach in Cloud Security for Data Engineers

Authors:
(1) Nora Schneider, Computer Science Department, ETH Zurich, Zurich, Switzerland (nschneide@student.ethz.ch);
(2) Shirin Goshtasbpour, Computer Science Department, ETH Zurich, Zurich, Switzerland and Swiss Data Science Center, Zurich, Switzerland (shirin.goshtasbpour@inf.ethz.ch);
(3) Fernando Perez-Cruz, Computer Science Department, ETH Zurich, Zurich, Switzerland and Swiss Data Science Center, Zurich, Switzerland (fernando.perezcruz@sdsc.ethz.ch). Table of Links Abstract and 1 Introduction 2 Background 2.1 Data Augmentation 2.2 Anchor Regression 3 Anchor Data Augmentation 3.1 Comparison to C-Mixup and 3.2 Preserving nonlinear data structure 3.3 Algorithm 4 Experiments and 4.1 Linear synthetic data 4.2 Housing nonlinear regression 4.3 In-distribution Generalization 4.4 Out-of-distribution Robustness 5 Conclusion, Broader Impact, and References A Additional information for Anchor Data Augmentation B Experiments 3 Anchor Data Augmentation In this section, we introduce Anchor Data Augmentation (ADA), a domain-independent data augmentation method inspired by AR. ADA does not require previous knowledge about the data invariances nor manually engineered transformations. As opposed to existing domain-agnostic data augmentation methods [10, 45, 46], we do not require training of an expensive generative model, and the augmentation only adds marginally to the computation complexity of the training. In addition, since ADA originates from a causal regression problem, it can be readily applied to regression problems. Even when ADA does not improve performance, its effect on performance remains minimal. This paper is available on arxiv under CC0 1.0 DEED license. Authors: (1) Nora Schneider, Computer Science Department, ETH Zurich, Zurich, Switzerland (nschneide@student.ethz.ch); (2) Shirin Goshtasbpour, Computer Science Department, ETH Zurich, Zurich, Switzerland and Swiss Data Science Center, Zurich, Switzerland (shirin.goshtasbpour@inf.ethz.ch); (3) Fernando Perez-Cruz, Computer Science Department, ETH Zurich, Zurich, Switzerland and Swiss Data Science Center, Zurich, Switzerland (fernando.perezcruz@sdsc.ethz.ch). Authors: Authors: (1) Nora Schneider, Computer Science Department, ETH Zurich, Zurich, Switzerland (nschneide@student.ethz.ch); (2) Shirin Goshtasbpour, Computer Science Department, ETH Zurich, Zurich, Switzerland and Swiss Data Science Center, Zurich, Switzerland (shirin.goshtasbpour@inf.ethz.ch); (3) Fernando Perez-Cruz, Computer Science Department, ETH Zurich, Zurich, Switzerland and Swiss Data Science Center, Zurich, Switzerland (fernando.perezcruz@sdsc.ethz.ch). Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Background 2.1 Data Augmentation 2.1 Data Augmentation 2.2 Anchor Regression 2.2 Anchor Regression 3 Anchor Data Augmentation 3 Anchor Data Augmentation 3.1 Comparison to C-Mixup and 3.2 Preserving nonlinear data structure 3.1 Comparison to C-Mixup and 3.2 Preserving nonlinear data structure 3.3 Algorithm 3.3 Algorithm 4 Experiments and 4.1 Linear synthetic data 4 Experiments and 4.1 Linear synthetic data 4.2 Housing nonlinear regression 4.2 Housing nonlinear regression 4.3 In-distribution Generalization 4.3 In-distribution Generalization 4.4 Out-of-distribution Robustness 4.4 Out-of-distribution Robustness 5 Conclusion, Broader Impact, and References 5 Conclusion, Broader Impact, and References A Additional information for Anchor Data Augmentation A Additional information for Anchor Data Augmentation B Experiments B Experiments 3 Anchor Data Augmentation In this section, we introduce Anchor Data Augmentation (ADA), a domain-independent data augmentation method inspired by AR. ADA does not require previous knowledge about the data invariances nor manually engineered transformations. As opposed to existing domain-agnostic data augmentation methods [10, 45, 46], we do not require training of an expensive generative model, and the augmentation only adds marginally to the computation complexity of the training. In addition, since ADA originates from a causal regression problem, it can be readily applied to regression problems. Even when ADA does not improve performance, its effect on performance remains minimal. This paper is available on arxiv under CC0 1.0 DEED license. This paper is available on arxiv under CC0 1.0 DEED license. available on arxiv

Anchor Data Augmentation (ADA): A Domain-Agnostic Method for Enhancing Regression Models

Authors: (1) William P. Wagner IV, Claremont Graduate University. Table of Links Abstract & Introduction Definition of Key Terms Fundamentals of Internet Operation Encapsulation Usage-Based Economic Models Net Neutrality Legal History Researcher Conclusions Areas for Further Exploration & References 3. Fundamentals of Internet Operation 3.1. Hardware Independence. As long as the hardware meets some basic technical requirements to communicate with its neighbors, there are no specific restrictions on hardware or communications medium. The Internet is Hardware Agnostic. All decisions about a given network’s hardware are left to the network administrator or owner to provide the best solution for their application. Copper, fiber, satellite or cellular – that decision is up to the network owner – the Internet does not care. 3.2. Network Independence. As long as the data packets arriving and leaving a network meet some basic technical requirements to communicate with their neighbors, there are no specific restrictions. The Internet does not care who the owner or administrator of a given network is or what topology that network employs – it will route packets based on “Best Effort” routing – the Internet is Network Agnostic. 3.3. Content Independence. As long as the data packets leaving a network meet some basic technical requirements to communicate with their neighbors, they will be prioritized by type not content or source. For example: A home user’s VoIP packet should almost always be more important than an FTP request from a big corporation, except on a subnet of only that company’s FTP servers. The Internet is necessarily Content Agnostic. This paper is under CC BY-NC-SA 4.0 DEED license. available on arxiv

The Agnostic Nature of the Internet: Understanding Hardware, Network, and Content Neutrality

Joining me in this interview is Christopher Goes, Co-founder of Namada, and we're going to discuss the challenges of integrating privacy with asset transfers, including both fungible and non-fungible tokens, and the technical solutions employed to maintain privacy across various asset types. We'll also touch on the role of zk-SNARK technology in enabling private transfers and the significance of making zero-knowledge proofs feasible on edge devices for wider adoption. Please introduce yourself, and tell us what you do. My name is Christopher; I primarily work on protocol and organizational design for Anoma & Namada at Heliax (which often, but not always, turn out to be two sides of the same coin). What were the most significant technical challenges in developing Namada, and how were they addressed? I would say that there were two significant technical challenges in developing Namada: Firstly, the design and implementation of the MASP circuit, which is quite internally intricate and must simultaneously satisfy safety requirements and provide a clear, efficient interface for users. Luckily, much of the work here was done in the past by the Electric Coin Company on Sapling, and we’ve been able to extend Sapling with multi-asset support and asset conversions while preserving all existing features (e.g., the key hierarchy). Second, the design and implementation of the shielded set rewards system must simultaneously satisfy safety and efficiency requirements and provide an incentive reward mechanism to recirculate the collective benefits of privacy back to individual contributors. As far as I am aware, this particular design is unique to Namada, and rewarding shielded assets carries unique technical challenges compared to reward systems in non-private DeFi contracts. Could you delve deeper into the concept of interchain asset-agnostic privacy? To Namada, interchain asset-agnostic privacy means providing privacy as a service to all existing and future assets which live on the interchain in a uniform way which is standardized in order to make users’ lives easier and which allows users of different assets to benefit from inclusion in a shared anonymity set. Privacy is a public good — the more users use privacy-preserving systems, the more privacy other users get — and Namada provides the protocol infrastructure to realize this privacy-preserving equilibrium. How can organizations balance the need for Asset-Agnostic Privacy with other objectives like transparency and efficiency? I do not think that these objectives are at odds with one another. Organizations and all users of Namada are free to disclose exactly what information they would like to exactly who they would like. Privacy in blockchain systems is less like drawing your curtains closed and more like choosing which group chat to send your messages to. Thanks to recent advances in applied ZK cryptography, using Namada for payments is as fast and as cheap as using a non-private chain. Could you elaborate on how Namada's proof-of-stake L1 architecture supports interchain asset-agnostic privacy? Namada supports interchain asset-agnostic privacy primarily through the IBC protocol, originally developed by the Cosmos ecosystem. Using IBC, other chains can permissionlessly connect to Namada, and users with assets on those chains can send their assets to Namada, transfer them privately, and send them back whenever they would like. Can you delve into the technical workings of the multi-asset shielded pool (MASP) circuit? How does it manage to provide privacy across different types of assets? The multi-asset shielded pool (MASP) provides privacy across different asset types by including in each note (UTXO) an asset type and the amount, owner, and other metadata. When transaction validity is checked, the MASP performs a multi-asset balance check, ensuring zero net balance independently across all asset types. Thanks to homomorphic value commitments with unique bases for each asset type, this is possible without any loss of privacy. For further information, see the MASP specification. How do asset-agnostic shielded transfers work, especially when dealing with both fungible and non-fungible tokens? Fungible and non-fungible tokens aren’t really that different in essence — modulo token-specific logic, a non-fungible token is merely a fungible token with a total supply of one unit, and this is how Namada represents NFTs. NFT-specific logic (e.g., breeding CryptoKitties) lives on the chain which issued the NFTs — Namada provides only privacy-preserving transfers, and standardizing the token logic in this fashion allows all fungible and non-fungible tokens to share the same anonymity set and provide maximum privacy for everyone involved. How does the interoperability with fast-finality chains via IBC and Ethereum via a trustless two-way bridge work practically? Practically, interoperability works via consensus verification: when the Namada validator set produces a block, a light client header and proof of this block is relayed to the other chain (say, Osmosis), which verifies it. This header can subsequently verify packets sent from Namada to Osmosis, such as a packet unshielding previously-shielded OSMO and sending it back to a user on Osmosis. The reverse direction works similarly, just with the two chains reversed. What role do novel zk-SNARKs play in enabling these private transfers? Namada relies on recent advances in zkSNARK technology — particularly, the Groth16 proof system — to provide efficient, compact proofs for the MASP circuits, which facilitate private transfers. Without zero-knowledge proof systems, this kind of practical privacy-preserving system would not be possible. Can you discuss the process of generating Zero-Knowledge Proofs (ZKPs) on edge devices and its importance for mainstream adoption? Users want payments which are cheap, fast, privacy-preserving, and accessible to them on the devices they already have — in particular, smartphones. If we want to bring privacy-preserving payment systems into the mainstream, we must meet users where they are, which will require proof generation support on phones. Luckily, lots of work in these directions has been done already, and much of the labor can be shared between projects. What are the next major milestones for Namada, and how do they align with the project's long-term strategy? As I write, Namada is conducting the Shielded Expedition, a massively multiplayer role-playing incentivized testnet, in preparation for the upcoming mainnet launch. Namada’s success depends on protocol correctness, usability, and user understanding, and the Shielded Expedition is designed to simultaneously test the former and facilitate the latter. How do you envision the future of privacy in blockchain technology? In my view, privacy — or programmable disclosure — is table-stakes for real-world usage of blockchain systems. The blockchain ecosystem has done a solid job funding applied cryptography efforts in recent years, which have made a lot of progress, but this has not yet quite materialized in cleanly integrated user-facing products. I think that we’re on the cusp of a phase change as the final technical and product pieces are now put together to make these systems ready for the mainstream.

Navigating Multichain Environments: The Role of Asset-Agnostic Privacy

Today, I decided to discuss languages and the importance of being language/architecture agnostic. Main Firstly, I observe that the backend world is rediscovering itself and criticizes microservices architecture. For example: Amazon Prime recently reverted to a monolithic structure. For me, the key idea is that we shouldn't blindly follow all mainstream trends. Rather, we should carefully decide which architecture to choose. Maybe the optimal decision for you is to either stick with a monolith or strike a balance between monolith and microservices. Following that, Tobi demonstrated that Rails can now be fast and exhibit high performance. Let's delve into it. Ruby on Rails is often regarded as a framework for startups, providing faster development of new features – a reputation that holds. However, when modern Ruby is combined with contemporary Rails, in capable hands, it can deliver speed even in the late stages of a product's life cycle. Now, let's consider PHP. This language has a bad reputation because it has a low border to enter and if you want to achieve fast and cheap product development you can do it with PHP. In my career, I saw that spaghetti code and mutant architecture can be produced even on fancy Golang or old-school C#/Java. Hence, projects with low maintainability can be agnostic to language and architecture. Examples A few years ago I joined a project that was rewritten from Java to Golang (monolith to microservices). However, the team decided to rewrite the code to another language at first and then extract microservices from it. I think that decision was made because Golang is fancy and it's cool to use it and microservices. However, progress was stopped when a few micro-services (like authorization and some other user services) were implemented. So, after some speed improvements from faster user login and increasing database health. We stumped with a huge half-monolith legacy in Golang that was hardly comprehensible by newbies in that project. Also, it was hard to sell continuous extraction to micro-services to businesses to whom a monolithic architecture and Golang would have been sufficient. Additionally, I can provide a positive example wherein a 10+ TB database was split into distinct databases, each serving specific purposes and supporting discrete microservices. Originally, it was a Ruby on Rails monolith, but due to load growth, certain components were extracted and transitioned into Golang microservices. These modifications caused development velocity improvement and added the ability to sustain high load Pieces of Advice Besides, you have to think over the use cases of your product. For example, maybe you want to analyze how your clients operate with your product. Or you need some monthly report that locks database tables for a long time. So, maybe you need to move analytical work to another service or database. And you always have to think and google about ready solutions . instead of reinventing the wheel Looking forward I hope that all mainstreams will be closely inspected before applying it to a real project. Because now, leading opinion can be just wrong and cause a lot of trouble with understanding and maintaining in the future. Conclusion Ultimately, I want to ask you: What will you choose or how will you adapt and work with various concepts of architectures and languages? What will be your choice?

Languages and Architecture — The Importance of Being Language/Architecture Agnostic

Introducing Nanc: A Backend-Agnostic CMS Perfect for Flutter Apps

ন্যান্সের সাথে পরিচয়: ফ্লটার অ্যাপের জন্য পারফেক্ট একটি ব্যাকএন্ড-অ্যাগনস্টিক সিএমএস

Wir stellen vor: Nanc: Ein Backend-unabhängiges CMS, perfekt für Flutter-Apps

Presentamos Nanc: un CMS independiente del backend perfecto para aplicaciones Flutter

Présentation de Nanc : un CMS indépendant du backend, parfait pour les applications Flutter

नैनक का परिचय: स्पंदन ऐप्स के लिए एक बैकएंड-एग्नोस्टिक सीएमएस परफेक्ट

Nanc の紹介: Flutter アプリに最適なバックエンドに依存しない CMS

Nanc 소개: Flutter 앱에 완벽한 백엔드 독립적 CMS

Apresentando o Nanc: um CMS independente de back-end perfeito para aplicativos Flutter

Nanc ile Tanışın: Flutter Uygulamaları için Mükemmel Bir Arka Uçtan Bağımsız CMS

Giới thiệu Nanc: Một CMS phụ trợ hoàn hảo cho các ứng dụng Flutter

介绍 Nanc：后端不可知的 CMS，非常适合 Flutter 应用程序

Learning how to query data from databases is not just for data scientists, it’s an essential skill for developers, too. Fortunately, NoSQL databases are schema-agnostic. You don’t need to be an expert in thousands of combinations of SQL, data types, and databases to master SQL queries. A good SQL agnostic parser will take care of all but impossible complex queries with ease. If you want to build full-stack applications, it’s important to create apps that use data from databases. Databases can store varies amounts of information in any format, such as text, images, or video. These data stores need to ‘talk’ with one another. Here the role of SQL agnostic parsing software becomes crucial, enabling databases to work among multiple systems rather than having to create a single customized solution from scratch that will degrade in effectiveness over time. SQL Agnostic Queries Make It Work As organizations intake and process an incoming flood of data and traffic, intelligent approaches to managing and querying a database need to be utilized. The proliferation of database types via open-source database software has contributed to the complexity. It’s not an Oracle-only world today. But even it were, there are still major syntax differences among even the simplest operations across all the major SQL databases. Even though they all implement the same SQL standard. A SQL agnostic query engine can account for all the differences. Database-agnostic querying is necessary to allow engineers to focus on the data they need instead of the data access mechanisms at work behind the scenes. To understand the power of SQL, it helps to think about how different types of queries function. For instance: Data Definition Language (DDL), Data Manipulation Language(DML), and Transactions Control Languag (TCL). These all work as an abstract version for various kinds of SQL statements that can be executed across databases and data types. Agnostic queries focus on answering the important question of ‘what’, rather than ‘how’ or ‘where’ information is stored. It also helps users retrieve information without being database literate. It makes SQL queries much less complex to write.  Users see a single unified representation of all the data available to them and get answers in an easy-to-use format. This doesn't mean that the data retrieved under these queries is not being managed or maintained properly. Data can be in a relational database, a graph, a foreign primary key value, or a time series.  Changes in the database schema are continuously tracked and recorded as they always have under standard SQL. Developers don’t need to know that ‘behind the scenes’ a good SQL agnostic query engine is tracking: The kind of database that houses the data, such as relational or key-value. The particular database instance such as a shard, or a partition The version of the database being used such as PostgeSQL or Microsoft SQL server. This is the magic and efficiency of using a database access tool that uses SQL agnostic queries. Example: Apache ShardingSphere is an open-source database ecosystem of tools and middleware that makes good use of existing database resources rather than building another new database technology. Apache ShardingSphere Apache ShardingSphere is an open-source ecosystem consisting of a set of distributed database solutions, including 3 independent products, JDBC, Proxy & Sidecar. They all provide functions of data scale out, distributed transaction, and distributed governance functions, applicable in a variety of situations such as Java isomorphism, heterogeneous language, and cloud native. Apache ShardingSphere is an Apache Top-Level project and is one of the most popular open-source big data projects. It was started about 5 years ago and now has 14K+ stars, over 300 contributors, and 5,100 forks in its community. Here is an exmaple of a query using ShardingSphere’s SQLParserEngine access tool: <insert code snippet here> The queries are more accessible for developers or anyone who wishes to extract information from the database since it's in a format that makes sense, rather than being all over the place with no standard for access. Advantages of SQL Agnostic Queries Backing For Unstructured Text The digital world is changing faster than ever before. It's hard to imagine that there could be so much unstructured data in the world, and yet we're barely scratching the surface. The reality is that 80% or more of all enterprise-level information comes from non-document sources, for example, text messages between employees on their phones. This means companies need a new approach if they want access to valuable business intelligence about what matters most - without having an entire research staff dedicated solely towards understanding where and how to access the data as it comes in. Numerous NoSQL data sets can deal with the ordering of unstructured text either as a local element (MarkLogic Server) or a coordinated arrangement of administrations, including Solr or Elasticsearch. A good SQL agnostic tool is like a Swiss Army knife for your data. It can answer the question of what, when and how much by translating unstructured or disparate elements into something useful that will get you closer to an answer than ever before. Managing Complexity Data scientists have persevered in finding ways to solve queries using the power of data for decades. The first step was optimizing databases, which was complicated and time-consuming, done ‘by hand' from memory - but could not handle large multi-table joins without breaking a sweat! Today's sharding solutions can split up huge tables across many databases so they don't get too heavy on any one server farm or database, but it makes queries more complex. NoSQL parsing software excels at hiding that complexity from users who simply want fast, insightful data retrieval. Reducing Cost The NoSQL data sets handle apportioning (sharding) of a single large, unorganized file across servers. This model supports less expensive commodity hardware and software which can result in significant cost savings for organizations. They won’t have to purchase additional server space or storage media with higher prices than is necessary. To Summarize SQL is the de facto language used to access many different databases and data types for all of today’s apps. SQL agnostic queries are now necessary to  write applications for multiple databases and data types. The syntax of SQL agnostic queries is very similar to standard SQL, so it's easy to learn and use. You don't have to switch between languages when you need a new database. It's easier than ever before because there are more tools like ShardingSphere available for making this process simple and efficient. Now, no matter what major changes come up with future versions of your favorite data platform, you'll always be able to keep using it without any issues, thanks to SQL agnostic queries.

SQL Queries: Why You Need SQL-Agnostic Parsing

Day by day, NFTs are making the case for cross-chain technology stronger. At the very least, they make a case for blockchain agnostic NFT marketplaces considerably more compelling for those in the know. As the market has grown, it has expanded across multiple chains from Ethereum, Binance Smart Chain, Avalanche, Solana, and layer two solutions such as Polygon and Immutable X. Putting gas fees aside, while collectors may or may not care which blockchain their is hosted on, those who collect NFTs for their resale value certainly do care how many pairs of eyes can view their collection. It's all about the potential size of the market, and right now, liquidity for NFT collections is highly segmented. It's not hard to imagine that an NFT tied to a single chain might have lower resale potential than one which can capture the interest of the broader industry, especially if the NFT is hosted on a marginally smaller native chain. NFT While the early NFT market and its growth was tied to the Ethereum network, NFTs, much like DeFi applications before them, are now branching out. One reason for the proliferation of NFTs across multiple chains is the inherent scaling problems of the Ethereum network. Unfortunately, attempts to make Ethereum more scalable have, as yet, seen relatively little success, allowing other chains to make moves on its crown. This is not the whole story, however. NFTs have become such an essential aspect of the wider crypto environment that it would seem almost careless for any major blockchain in the sector not to have at least a few good NFT projects in its stable. Cheaper and Faster While Ethereum remains the king, its grip on the market appears to be weakening. Binance Smart Chain is one of the rivals leading the charge, having now surpassed . The fact that BSC is a fork of Ethereum makes it relatively easy for projects to port from one chain to the other - and with far lower fees, some retail investors feel better cared for on the rival chain. However, BSC is not the only contender with which Ethereum must reckon with. 100 million unique addresses Avalanche, which focuses on speed and low transaction costs, is another one of the emerging competitors seeking to capture a share of the NFT market. The blockchain is now attempting to eat into the market share of the big players. This need for speed has also helped fuel the growth of the aforementioned Binance Smart Chain, as well as Solana and layer two solutions, including Polygon. One of the NFT marketplaces that has capitalized on this drive towards faster and cheaper chains has been . The cross-chain, blockchain agnostic marketplace has successfully positioned itself to become the #1 destination for Avalanche NFTs both by users and trading volume. Solana will become the fifth blockchain the market will support, joining Ethereum, Polygon, Avalanche, and Binance Smart Chain. A 6th, 7th, and 8th chain is expected to follow shortly afterward. NFTrade , one of the first-movers in the NFT market, is also extending cross-chain support - albeit at a more leisurely pace. The marketplace which has been at the forefront of the NFT market on Ethereum also supports Polygon and Klatyn. OpenSea While a number of marketplaces remain tied to a single chain, there are increasing numbers going multi-chain. This is a solid long-term strategy for those who wish to future-proof themselves, especially as cross-chain bridges make native network assets much more fluid. What Matters One of the big questions which market analysts must grapple with is how much users care about decentralization and blockchain fundamentals versus cost, transaction speeds, and other concerns. The top chains have all taken different approaches to address these problems. Solana is one of the fastest chains on the market with 400 millisecond block times, but it is also highly centralized. In September, the Solana mainnet was overloaded and even needed to be rebooted. The implication of such an action is clear; a network that can be stopped and restarted by a controlling entity is arguably not a distributed blockchain at all. Despite this, Solana is also growing considerably in popularity, especially in the NFT sector. For years, a prevailing belief in the blockchain sector has been that decentralization is one of the most important factors (if not the most important), but for some users, it seems that this is not the full case. While Ethereum has grappled for years with Eth2 and moving to Proof-of-Stake, other chains are quite happy to take advantage of this gap of capabilities by providing more centralized solutions. NFT Positive, Blockchain Agnostic The incredible growth of the NFT market and its propagation on multiple chains shows the impending need for multi-chain integration. Cross-chain technology is one way to deal with the segmentation of NFTs across multiple chains; blockchain agnostic marketplaces are another. With increasing numbers of chains and options for creators, collectors, and users, NFTs need cross-chain and blockchain agnostic marketplaces. In any case, the NFT market appears to have a truly bright future ahead of it - on many different blockchains. This article is part of The hosted by HackerNoon in partnership with Gaming Metaverse Writing Contest The Sandbox . Submit your today for your chance to win up to $2000. #gaming-metaverse story

Why NFTs Need Cross-Chain Technology and Blockchain Agnostic Markets

For years now, the attention of the world has remained focused on the financial implications of the cryptocurrency revolution. And that makes sense because the crypto market has grown to a valuation of over $2 trillion. But if you remember a few years ago, the financial aspect of cryptocurrencies wasn't what had most people excited. It was the blockchain. Early on, the business world seized on blockchain technology as the true killer app to emerge from the birth of Bitcoin and its descendants. But as time went on it started to become clear that blockchain wasn't the secure data storage panacea that it was made out to be. And that reality has hampered most businesses' attempts to use it in an effective way. And here at , we struggled with the limitations of the blockchain as we sought to build out our RegTech platform. But then, we realized that there was a better way. A way that leveraged the unique security offered by blockchain technology while driving down data storage costs. And even better, it created an ERC20-based token transaction system that ensured auditability for our users' data. Here's an overview of what we're building and why we think it's the future of RegTech. Sekuritance What is RegTech? To understand the ecosystem we're developing, you must first understand the services it's meant to facilitate. As a RegTech innovator, we provide various services that help businesses comply with whatever regulations they're subject to. For that reason, we were looking to offer PCI-compliant data storage, KYC and AML functions, and risk-scoring for businesses. But unlike incumbent providers of such services, we wanted to offer a unified ecosystem that brought our suite of services together with third-party integration opportunities. And that's where the SKRT utility token comes into play. A Blockchain-Agnostic Platform The SKRT utility token is the transaction medium that unites the Sekuritance RegTech Platform. It's how we track all data vault queries, checks, transactions, traces, and findings. It ties our non-blockchain technology stack to an auditable blockchain that guarantees an audit trail for every service we offer. And our customers use it as the primary means of accessing our services. At present, the SKRT smart contract operates on top of the Ethereum ERC-20 protocol to establish liquidity and a trading regime. But we've built the system to be blockchain-agnostic. That means it can run on Bitcoin, Ethereum, or private permissioned blockchains like Hyperledger. We think that's a critical feature – because the Achilles heel of all blockchain projects is forced obsolescence when newer, better protocols emerge. An All-In-One Solution By establishing the SKRT token and its underlying smart contract, we've been able to create a hybrid system we call the SekurSuite™ product suite. It offloads the payment processing and auditing functionalities to the SKRT ecosystem while the underlying data storage and processing take place in a distributed cloud setting. In this way, we were able to leverage commodity hardened data storage facilities to create a PCI-compliant infrastructure, while creating an immutable access trail via SKRT. But as we've mentioned, we had another motivation for tying our platform together with the SKRT utility token. By creating a token economy, we hope to encourage the adoption of our product suite as well as to encourage third-party developers to publish their products within our ultra-secure app ecosystem. That will allow us to grow our ecosystem into a true hub for RegTech innovation that will push the boundaries of what's possible with current technologies. It will also foster the development of successor technologies by bringing the best and brightest RegTech and blockchain development minds under a single roof. A New Identity Data Model We've also created a program called the Partner Network Rewards System. It's designed to award points to individuals and businesses every time their information is processed by us or any of our partner data providers. Then, the individuals or businesses can claim their data profiles and use their token balance for services through our network – or they can withdraw them to use as they see fit. But most importantly, those profiles will allow for user feedback. This means those who claim their profile will be able to have some direct input into how others view their data. For example, they'll have the opportunity to comment on or explain negative information in their profile, as well as highlight positive information. In a world where we have little control over the information available about us online, and where others' impressions of that data shape our destinies – we see this as a revolutionary feature. The Future of RegTech Going forward, we're looking forward to continuing to refine the SKRT ecosystem and we invite interested developers to explore the opportunities it provides. Together, we hope to create the foundations for a new secure data and RegTech infrastructure to meet the demands of the 21st century. And in the process, we want to leverage blockchain to create the true killer app everyone expected it to spawn. We think we're on the right track. And with your help, we're going to prove it. Featured image: WrightStudio / Adobe Stock

RegTech Platform That is Blockchain Agnostic? You Heard It Right

Three years ago, I released an as a proof of concept to demonstrate an approach to build real-time CRUD applications using , and Google’s framework. Although it received a lot of attention initially, that approach did not see any serious adoption and so I stopped thinking about it for a while. open source boilerplate application RethinkDB SocketCluster Polymer Recently, though, I’ve been feeling frustrated by the direction of progress in front end development and so I decided to give this idea another shot. Polymer has always been a relatively niche framework; maybe those libraries just needed broader framework support? I rewrote the Polymer components in a framework-agnostic way and ported the entire app to Vue. sc-crud-sample The new and modules can now work with any front end framework including Vue, Polymer, React and Angular. SCModel SCCollection Please check out — Feedback is welcome. https://github.com/SocketCluster/sc-crud-sample Also, if you want more background info about this work, please check out my previous articles on the subject: _These days, it’s practically impossible to talk about REST without also talking about HTTP — The two concepts are…_hackernoon.com REST over WebSockets instead of HTTP _Recently, I wrote an article about using REST over WebSockets which generated quite a bit of interest—So I would like…_hackernoon.com An approach to real-time CRUD

Framework-agnostic real-time CRUD modules with Vue sample app

An emerging trend in design systems is the idea of . There’s clearly a difference between , and the used in modern frontend frameworks like , Angular and Vue. separating your core HTML & CSS from the current JavaScript framework presentational JavaScript JavaScript React Seeing the value of this separation in your design system can be challenging when you’re working exclusively with React, JSS, or any other JavaScript that flips the traditional separation of concerns on its side: by Cristiano Rastelli Slidedeck Jeremy Keith recently described the separation of concerns in the above graphic using a : fantastic metaphor Presenting them as though they were binary choices is like saying “I used to eat Italian food, but now I drink Italian wine.” — Jeremy Keith It would seem at first glance that our HTML & CSS are destined to be eternally coupled with their JavaScript at the component level. Let’s take a look at you’d want to create this separation at the design system level, and at how a few leading companies are publicly sharing their practice. why So why would you want to separate core HTML & CSS in your design system? The way I see it, there are three potential driving factors: Product Variety JavaScript Framework Churn Predicting the Future Product Variety First, the need to keep HTML & CSS separate will directly depend on the number of products you have to support with your design system, and the underlying tech stack of each. The wider variety (one uses React, one uses Angular; one uses Sass, another uses vanilla CSS, etc.), the greater need to have your core components available as HTML & CSS. JavaScript Framework Churn I don’t have to convince you that JavaScript frameworks are being released at an exponential rate. What’s relevant here is the idea that if you incorporate a JavaScript framework at the design system level, the amount of work you’re creating for your team is also exponential. Say your team decides to incorporate Vue in the design system. You’ve got substantial buy-in that the company is standardizing on Vue. Fantastic. Then, 3 months down the road, one team decides they don’t want to use Vue, but prefer React. They’re stuck either building things from scratch, or figuring out how to get React to work with Vue. Extrapolate that out across products and, as companies grow, multiple lines of business, and you’ll understand the need. Even when companies standardize, there will always be outliers on different teams. Larger still is the issue of rewriting everything in your design system when the Vue comes along. Will that be in 3 years? 3 months? next 3 days? Predicting the Future I’m not good at this. If you’re Merlin, you can stop reading here. Still here? Okay, let’s look at some best practice. Salesforce Lightning Design System The Salesforce Lightning design system is becoming the poster child of modern frontend design systems. Jina Anne not only created it, she’s gone on to create the first 100% . She wrote chapter 2 of . design systems focused conference InVision’s Design Systems Handbook The Salesforce team has to support a variety of platforms, and maintain consistency across them. In addition to web projects, Lightning also supports . Here’s what the system’s FAQ has to say about JavaScript: iOS, Android and Windows Phone Is any JavaScript included as part of the framework? No. The Salesforce Lightning Design System is a pure CSS framework that you can use with any front-end development framework you’d like, including Salesforce-specific technologies such as Visualforce and Lightning, as well as third-party frameworks like React or Angular. And that’s where it gets interesting. The Salesforce team is able to take the core HTML & CSS Lightning components and create a separate React-based project from them. Here’s what Jina had to say about that, on the : Style Guide Podcast “Yeah, we do not have any JavaScript in our Open Source Design System and that is because once you introduce that, it is really difficult to stay agnostic; you basically introduce opinion into the stack. However, internally, and this actually might be Open Sourced as well; we do have a React version of our Design System that some of our smaller product teams use and then Lightning components come with all that baked in so if a developer at Salesforce is using a Lightning component, they don’t have to worry about building that because somebody’s already built that into the component.” — Jina Anne, on the Style Guide Podcast Lucky us, that React project has already been made public: The Lightning Design system is an example of an entire suite of components written in vanilla HTML & CSS, with fantastic documentation in their “Design Guidelines”, and a focus on accessibility. This enables their team to build a React version for React-based projects, but that project starts with all the core best practice built in. Maybe I’m just giddy because Lightning pulls off so well what Brad originally described in Atomic Design: One of the biggest advantages of establishing a thoughtful design system is that it allows organizations to scale best practices. If all those best practices — responsiveness, accessibility, performance, UX, ergonomics, and so on — are baked into the system, users can simply plug in the patterns and reap the rewards. This means design system users don’t have to be senior-level designers or developers to produce good work; the design system serves as a quality control vehicle that helps users apply best practices regardless of each individual’s skill level. — Brad Frost, Atomic Design Microsoft Fabric Design System Micah Godbolt wrote the fantastic , which included some very neat ideas for abstracting parts of your design system into JSON, what he referred to as “schema-driven design”. Since publication of the book, Micah has been working for Microsoft, on the . Frontend Architecture for Design Systems Fabric design system The Fabric design system is built to support core web, React, Angular and iOS. The “get started” process offers the following options: CSS The Fabric core contains very low-level CSS and design elements that are easily shared cross-project: What I’m really excited about is the “theme generator” in Fabric’s core — in the “output” section we have theme values available as JSON, Sass and PowerShell. This indicates that the majority of Fabric’s core is available as data that can be translated to different formats, likely the driving force behind being able to support React, Angular and iOS: React Angular The offers a great detailed explanation of what their team is able to accomplish, and some of the challenges. video referenced in the Angular docs iOS Fabric is different than Lightning in that its primarily exposing low-level pieces in its core (icons, colors, layout, etc.), as opposed to having what Lightning refers to as “Component Blueprints” for all the components it offers. I think if teams are starting off on the path of abstracting core pieces from their JavaScript layer, the Fabric example would probably be a good place to start. You can investigate further in the Fabric codebase, and where refers to a few of the methods used in Fabric. Micah also has a blog post Github Primer Design System Remember that design systems conference I mentioned earlier? Yeah, last year , and I naturally investigated to see if there was anything public related to her work and the team at GitHub’s approach to frontend design systems. Lucky for us, there’s a shiny new docs site: Diana Mounter gave a great presentation on color Primer is very well fleshed out, containing core principles (color, layout, typography, etc.), utility classes, and a very well documented component library. What’s noticeably lacking here is JavaScript. The is perhaps the best example I’ve come across of explaining to a development team the methodology behind a HTML/CSS framework as the foundation for your design system. There’s a clear focus on favoring ease of understanding over other software principles like DRY. principles documentation The Nutshell Its clear that there’s value in separating your HTML & CSS from your JavaScript at the design system level. We ended up here because when MVC-style JS frameworks were introduced, they were treated as another dependency, when they are patently a fundamental shift in the role JavaScript plays. jQuery was the last popular “presentational” JavaScript library, which could be added as a dependency and used whenever functionality was required. Jeremy Keith described Angular like this: Well, yes, Angular a front-end framework, but conceptually and philosophically it’s much more like a back-end framework (actually, I think it’s conceptually closest to a native SDK; something more akin to writing iOS or Android apps, while ). technically is others compare it to ASP.NET — Jeremy Keith MVC-style JavaScript frameworks are so different from layering in presentational JavaScript that we needed to change how we think about writing code for the web. In the above examples, we’ve seen how the emerging trend is to also change how we think about our design systems at the core level. React and Angular are treated as more similar to iOS in Fabric because they are just that: vastly different from the low-level pieces provided in Fabric’s core. I can envision a core HTML/CSS design system which not only contains all the low-level elements (colors, typography, etc.), and components built in vanilla HTML/CSS, but additionally examples of the components in the different templating languages or JS frameworks used in products the system supports. Similar to how . Pattern Lab uses tabs for HTML and Mustache Whichever path you decide to take will very much depend on your product variety mentioned above. The earlier you decide to abstract low-level pieces at the design system level will be the day you start saving massive amounts of work for your team in the long run.

Design Systems Should be JavaScript Framework Agnostic

It’s been nearly a decade since blockchain was invented with the promise of providing an alternative to the old monetary system and the ability to make trustless transactions online without a centralized third party. The industry has made a lot of progress especially in the last few years with the recent ICO craze, however ubiquity of merchants who actually accept crypto has been far from reached outside of a few coffee shops and online merchants here and there. For example, according to coinmap.org there are only around 13K venues dealing with cryptocurrency, including ATMs, globally. http://coinmap.org/#/world/9.27562218/34.45312500/2 For example, in Calgary, Canada there are dozens of Bitcoin ATMs including one that’s just a few hundred meters of where I am writing this, but the strange thing is that shop doesn’t actually accept crypto as payment. And there are only a handful of places that do accept cryptocurrency in the city. Clearly there is demand for cryptocurrency on the consumer end of things otherwise these shops wouldn’t have ATMs that sell Bitcoin and at a ~15% markup at that. The lack of merchant adoption is really hindering growth in the space. The more places you can use cryptocurrency the more value it has and the more reason to spend on a day to day or month to month basis. The more participants the more stable the price and the higher the value. Here are two of the major problems hindering adoption 1. Volatility This is one of the biggest issues as merchants need to make sure that their assets are relatively stable, as they need to be able to predict expenses, pay employees, taxes, and make orders well in advance. This is especially true for medium to small businesses who have tighter profit margins and don’t have the benefits of economy of scale. A small business owner can’t wait months or years for the next bubble to recoup the losses from volatility. a dollar today is a dollar tomorrow This is also a problem for regular consumers. Consumers should be able to benefit from this new technology without taking undue risk from an immature market. Most people realistically don’t have the time or energy to be playing in the volatile crypto market. Saving a few dollars on transaction fees isn’t worth losing 70% of your money over just a few months, especially in first world countries where risk of hyperinflation or government forfeiture en masse is very low. The juice isn’t worth the squeeze as they say. 2. Technological Overhead (Complexity) Not only do merchants have to figure out how to use crypto and make special accommodations for it, they likely have to be constantly selling on an exchange to protect themselves from volatility. This isn’t really worth it in most cases just for the small percentage of people that might pay with crypto. What’s more is “accepting crypto” can actually be quite overwhelming for merchants as the number of new tokens and coins is proliferating each day. . Many of which would have to sell on different exchanges or use different platforms and have a lot of friction before getting back to a local or preferred currency. There are well over 1800 crypto assets to choose from and almost each one would take some sort of special technological integration and implementation Each person I know who owns crypytocurrency has a completely different portfolio so even if we imagine crypto reaches 100% saturation for consumers, each person coming into a coffee shop would be paying with a different currency. By the end of the day the coffee shop owner is going to have to potentially deal with hundreds of different currencies, most of which they won’t want. You can imagine how many stickers you would need on your window if you were to accept dozens or hundreds of currencies and how quickly this would get out of hand trying to accept so many. (Of course stickers on your window is the most trivial issues in implementing multiple cryptocurrencies, but it’s a good visual representation of the larger problem.) . We badly need a scaleable solution for doing commerce in many or nearly unlimited currencies pairs. Merchants should only need one sticker that reads “CryptoCurrency Accepted Here” Solution: Transaction Agnosticism Transaction agnosticism is the ability for Alice to pay Bob with whatever asset(s) she wants, and for Bob to instantly receive whatever asset(s) he prefers as payment. And so Bob doesn’t even need to know or care what Alice’s personal preferences or assets are and vice versa. Some solutions to achieve this could be with . Although this will probably be part of the solution in the future its not a complete or efficient solution. The reason being that liquidity between arbitrary and arbitrary , in a lot of cases will be very low and there is still a good chance of slippage even after finding a path as well as some computational overhead. pathfinding currency A currency B In order to have transaction agnosticism by swapping arbitrary pairs directly for one another you would need to be sure that there is sufficient liquidity between pairs, where is the number of supported currencies customers might use and is the currencies that the merchant wants to accept. However since you likely want to be using a platform that can be used ubiquitously, allowing any individual merchant to easily change and accept different currencies the platform should be able to handle currency pairs just to be safe. N^X N X N^(N — 1) . If we take the numbers from coinmarketcap.com, there are at least ~1800 cryptocurrencies and lots more on the way or yet to be listed. For this amount there would need to be sufficient liquidity between ~3,238,200 possible pairs As more assets hit the market the problem of liquidity will increase exponentially, as most financial instruments will be migrating to the blockchain at some point in the future. Without a solution it will render most assets unspendable in any sort of brick and mortar type situation or even on lots of online shops, devaluing the industry as a whole since you can only actually use the coins or tokens in very limited capacities. Illiquid pairs could cause the parties to either have to wait a long time or pay a bad price for the transaction nullifying blockchains benefits of cheap and/or fast transactions compared to traditional means. A solution is to use currency as a settlement or bridge currency. Such a transaction would look like this; swap for , swap for , send to merchant (all automatically on the back-end of course). This way we only need to make sure currencies have liquidity with Using a bridge currency means you only have to worry about liquidity with possible pairs, drastically reducing problems with liquidity as it scales linearly as more currencies are supported instead of exponentially. X A X X B B X. N-1 We at are working on making the process even smoother by adding transactions, meaning we maintain liquidity pools to allow us to simultaneously place a sell order: for , while placing a buy order: for . So even if the we can’t sell for a reasonable price instantly, or buy instantly, the merchant doesn’t actually need to wait for that to happen and they still receive the payment in their preferred currency Just In Time, reducing price slippage of the exchange and increasing speed. TillBilly “Just In Time” A X X B A B Illustration of proposed TillBilly’s Point of Sale Transaction Model Ultimately with “Just In Time” transactions we can wait and take the risk to make a swap at a good price whereas a consumer and merchant shouldn’t be bothered to even think about things like liquidity or even form of payment with modern technology. There are far more important things in peoples lives to ponder. It is becoming evident that the next big movement within the blockchain space is tokenization and securitization. With transaction agnosticism this opens up some interesting opportunities for people in their choice of currency. For example cryptocurrency users will have the ability to hold asset backed tokens, like a token backed by an apple orchard issued by a company like Smartlands or even something like Amazon stocks. You would then be able to use that to buy groceries from the supermarket or coffee from your favorite coffee shop. Unlike traditional stocks or commodities you would always be liquid and fungible, no need to sell a large chunk of your investment if you’re short on cash that month. Just spend the amount you need to, when you need to. This way you can keep the upsides of assets, like stability, appreciation., or stock dividends without the downsides of low liquidity and friction caused by fees when cashing out. This will create great investment opportunities for people all over the world as it allows them to have a wide and diversified portfolio of assets that are not linked directly to their local economy or politics in the way that national currencies so often are at the mercy of. And unlike most traditional forms of investing the bar for entry is almost nonexistent, the minimum amount needed to invest is enough to cover transaction fees. Such optionality for nearly unlimited payment types within a single or few platforms will certainly spur adoption of blockchain particularly because it gives users the option to use relatively stable assets like fiat or commodities ect and easily switch between them. For crypto to grow it doesn’t need random partnerships or acceptance of random currencies on different sites, but a generalized, currency agnostic, payment solution for both brick and mortar and online shops. Original Author: Jo , community manager at TillBilly

Transaction Agnosticism: A path to Cryptocurrency Mass Adoption

Vendor lock-in — being locked into a specific cloud provider through the use of services specific to that cloud provider I’ve been working with my head in the cloud for the past 2 years now and one thing is pretty certain — Cloud-agnostic is a , at least in a conventional sense. Architecture Myth Overly-generic stock photo to represent vendor lock-in. In this article, I’ll be discussing the current state of cloud services and how being truly cloud-agnostic is not only something we can’t quite achieve, but why it isn’t necessarily something you may want to achieve. Dockerizing Your Application Our first port of call is Docker. I am a seriously massive advocate for containerization technologies such as Docker and have recently given talks demonstrating how these technologies could massively improve the way we build and deploy software. One of the key advantages of Docker that I like to highlight, is its ability to reduce the friction when it comes to migrating from one cloud service provider to another. You specify everything that your application needs to run in terms of runtime, environment variables and any setup scripts and you are able to start said container almost everywhere. I say *almost* as there are some very minor exceptions such as Windows-only images etc. This essentially means that the Python application we need to run can be containerized and dropped into any managed container service such as ECS. The underlying container platform will then manage the container lifecycle and provide additional benefits such as logging and monitoring should we wish. The underlying OCI-compliant container runtime will then handle any discrepancies between hosts and run your application as you intended it to run with minimal fuss. When it comes to trying to avoid vendor lock-in, containerization is an incredibly low-hanging fruit. The Growth of Terraform Terraform is the latest “shiny thing” that I’ve had the pleasure of playing with in my travels and I’m very much a fan of the CLI and how it lets you define your infrastructure as code. You can specify the provider you wish to use as well as the access keys and secret keys and when you run , the CLI will the go away and spin up the appropriate infrastructure for you to then deploy on top of: terraform apply If you subsequently need to tear down your infrastructure, you can with ease using the command, which is a command that sounds freaking awesome. terraform destroy If you are looking for a good reference manual on Terraform, then I highly recommend the Yevgeniy’s book: _Terraform has emerged as a key player in the DevOps world for defining, launching, and managing infrastructure as code…_amzn.to Terraform: Up and Running: Writing Infrastructure as Code Now, this is great for spinning up your system’s required infrastructure on a specific cloud provider, but this means we will have to write multiple Terraform scripts for each of the distinct cloud providers. This isn’t entirely out of the question, but we then have to maintain these scripts going forward in such a manner that we can ensure their correctness should we need to fail over to a given provider. There also needs to be a certain level of knowledge regarding the underlying platform in order to be explicit about what instance types you are spinning up or what size these instances need to be. From an application developer’s point-of-view, you may not necessarily have to, or want to, care about what AMI you are creating your t2.micro instances from, you may just care about developing your application and having somewhere stable to run said application so that it can serve your paying customers. The Nuances between Services One of the potential solutions to this that I have seen mentioned is to implement a layer of abstraction above all cloud service providers and essentially define your application’s underlying infrastructure as a series of blocks. Essentially an extension on top of Terraform that would have the ability to spin up cloud-platform agnostic infrastructure. This Terraform-esque abstraction layer would then have the job of interfacing with whichever cloud platform it so chooses and spinning up said infrastructure. This would let you define say, a simple web app, as 1x “Load Balancer” and 2x “Small Server”. To some, this seems like the obvious solution, but when it comes to implementing this abstraction layer and dealing with the nuances between how different services are exposed, this becomes next to impossible. Trying to handle this at an application level would be next to impossible and your codebase would be riddled with hundreds, if not thousands of conditional flags in order to cover a large number of services. Could we abstract this into a service-broker or SDK? Maybe, but it just shifts our initial problem left. Kubernetes So, Kubernetes is incredible and I’ve discussed some of its potential benefits for Enterprise environments here: _According to Betteridge’s law of Headlines, the answer to the above headline would be a resounding ‘no’. However, in…_medium.com Can Kubernetes Revolutionize The Enterprise World? Kubernetes, in combination with Docker and Terraform, may actually be somewhat of a good answer when it comes to attempting to be agnostic. It’s also something that has been explored by a number of different companies and individuals and we are starting to see managed solutions come about such as Triton. _Tutorial for getting started with Triton Multi-Cloud Kubernetes. You'll understand how Triton Kubernetes-Rancher, and…_www.joyent.com Multi-cloud Kubernetes with Triton | Joyent These federated kube-clusters (klusters?) can then be treated as an amalgamous lump of compute that can then run and manage any container-based applications you may wish to run. The biggest advantage of this is that application developers need just worry about how they want their application to look and does abstract away the underlying infrastructure. But then this isn’t perfect, these services are still very young and have yet to hit the mainstream. It’ll most likely be a while before these services become mature enough to be picked up by larger enterprises. The Value of Managed Services Vendor lock-in — The art of leveraging specific services from a cloud provider in order to provide value to clients One of the biggest things I’ve seen people disregard when talking about vendor lock-in, is the fact that there is a huge advantage in leveraging these managed services for your own gain. When considering the use of these managed services, development teams need to weigh up how much of an advantage these services provide and consider how much of a disadvantage they are subsequently put at by locking themselves into said service. Building a service like this yourself would take years of work. With managed services, it can be done in a day! If service X from AWS, or Azure, or GCP can provide you a massive advantage over your competition then it surely doesn’t make sense to avoid using said service because of vendor lock-in. If anything, by actively leveraging these services you may find yourself able to iterate faster with new ideas and push the bounds of what we are currently capable of. If we consider the likes of AWS’ RDS, if we were to attempt to host our own database instance on an EC2 instance and manage this ourselves, we would end up spending hundreds of hours attempting to achieve the same level of resiliency, or performance that an out-of-the-box RDS offering provides. This article goes into a more detailed comparison for self-hosted vs RDS: _Database-as-a-service (DBaaS) is a cloud service model that enables users to use databases without configuring physical…_cloud.netapp.com MySQL Database Migration: EC2 Hosted vs. Amazon RDS Conclusion Whilst vendor lock-in is something you should consider when designing and developing your applications, I would put far more stock in the advantage of using said managed service as long as you understand its constraints fully. Being mindful of the fact that you are relying heavily on these services is far more valuable in the long run and can give you an edge should you leverage these in appropriate ways. somewhat As it stands however, being able to remain fully agnostic of cloud provider is just not possible. More work will have to be done in order to provide that level of abstraction above said provider and at this point it’s just turtles all the way down. Hopefully, you enjoyed these ramblings, if you don’t agree with anything or would like to add to this then please feel free to let me know in the comments section below! If you’d like to support me then feel free to follow me on twitter: @Elliot_f or subscribe to my YouTube channel! _Hi Guys! Welcome to my channel, TutorialEdge! This is where I'll be posting all of my video tutorials on all of the…_youtube.com TutorialEdge N.B — This article was peer-reviewed by my good friend so please @ him if you see any mistakes! Alan Reid

Cloud Agnostic Architecture is a Myth

“No problem thumbs up” via Giphy Nowadays, unit testing is a required skill for a Software Engineer. But, many of them don’t know that, and think that it’s something from another world. The fact is: Unit tests are just some lines of code wrote to ensure that other lines of codes are working as expected. These tested code lines are the features of your application. That’s it, there’s no kind of magic. I don’t wanna scare anyone. So, I’ll not use libraries, standards, classes, and keywords. I’ll try to explain unit testing in a simplistic way. Using pure code, without any importing of classes or libraries. The examples are in Python. Motivation Why we have to do unit tests? Unit tests are the simplest way to avoid or warning you of problems. Like when some code change breaks another functionality of the application. Let’s see a code for example: As we expected, the function works fine. Here is examples of some values and the respective returns: get_company_as_string In [1]: get_company_as_string('Samsung')Out[1]: 'Name: Samsung | Founders: Lee Byung-chul.' In [2]: get_company_as_string('Apple Inc.')Out[2]: 'Name: Apple Inc. | Founders: Steve Jobs, Steve Wozniak and Ronald Wayne.' In [3]: get_company_as_string('Microsoft')Out[3]: 'Name: Microsoft | Founders: Bill Gates, Paul Allen.' In [4]: get_company_as_string('XPTO') In [5]: But, if, we need to create a function that return the sum of ages of founders on the company foundation? So, we need to adjust the data structure so that the list of founders keep their age. After that, we have the following code. That is the base of the previous code, with a modification in founders structure and a newer function: And, when we run our new function, we have this results: In [1]: get_sum_ages_of_company_founders('Samsung')Out[1]: 26 In [2]: get_sum_ages_of_company_founders('Apple Inc.')Out[3]: 87 In [3]: get_sum_ages_of_company_founders('Microsoft')Out[3]: 41 In [4]: get_sum_ages_of_company_founders('XPTO') In [5]: So, the implementation of the new feature was a success, and we can deploy a new version of our application. Is it? No no no! Let’s execute the function . Notice that we didn’t change that function, and we already verified that it was working fine. Let’s see if it keeps going well: get_company_as_string In [6]: get_company_as_string('Apple Inc.')Out[6]: "Name: Apple Inc. | Founders: [{'name': 'Steve Jobs', 'age': 21}, {'name': 'Steve Wozniak', 'age': 25}, {'name': 'Ronald Wayne', 'age': 41}]." Surprise! The results are not the same, and the function doesn’t work anymore as we expected. Yes, it’s a simple example, but, imagine if our application had more than 100 or 1000 functions. How can we ensure that all functions keep working as we expect? Or get an alert when something goes wrong? You know the answer: unit tests! Assert/Expect That’s the base, and probably, the simplest thing of unit testing. It’s kind of a little talk with the function. We tell something to the function and verify if the response is what it should be. For example, how can we test a function that sums two numbers? Give to the function two numbers. Compare the return with the sum of those two numbers, that we already know. If the function is wrong, the comparison will be fail. We know that 2 + 2 is 4, -3+-2 is -5 and -1+3 is 2. So, if we send 2 and 2 to the function, we expect receive 4. If we send -3 and -2, we expect receive -5 and, if we send -1 and 3, we expect receive 2. If one of them is different of what we expect, the function is wrong. It’s simple, right? Let’s see how the unit testing could have helped us in our previous refactoring. Remembering: I’m not using any specific library or class. I wanna simplify the explanation and keep it agnostic. In the old version of application code, the function was working fine. At this point we knew some stuff: get_company_as_string If we tell to the function, we expect to receive ‘Apple Inc.’ ‘Name: Apple Inc. | Founders: Steve Jobs, Steve Wozniak and Ronald Wayne.’, and If we tell we expect to receive . ‘XPTO’ None So, in the coding-way: get_company_as_string(‘Apple Inc.’) should be equal to ‘Name: Apple Inc. | Founders: Steve Jobs, Steve Wozniak and Ronald Wayne.’ , and should be equal to . get_company_as_string(‘XPTO’) None Here is the code: Minimalist test example without classes and libraries … and the execution of tests: In [1]:...: if get_company_as_string('Apple Inc.') == 'Name: Apple Inc. | Founders: Steve Jobs, Steve Wozniak...:  and Ronald Wayne.':...:     print('Success on test #1.')...: else:...:     print('Error on test #1.')...:...: if get_company_as_string('XPTO') is None:...:     print('Success on test #2.')...: else:...:     print('Error on test #2.')...:Success on test #1.Success on test #2. Now, we have the code that ensure the proper functioning of the function. So, we can test the functionality after the implementation of a new feature. In [1]:...: if get_company_as_string('Apple Inc.') == 'Name: Apple Inc. | Founders: Steve Jobs, Steve Woznia...: k and Ronald Wayne.':...:     print('Success on test #1.')...: else:...:     print('Error on test #1.')...:...: if get_company_as_string('XPTO') is None:...:     print('Success on test #2.')...: else:...:     print('Error on test #2.')...:Error on test #1.Success on test #2. We got an error. Or would that be a success? The changes that we did when implemented new feature broke the function But, our tests could identify this problem, and now, we can fix that. get_company_as_string. So, with a little change in the function, we can verify that all tests passed and ensure the quality of our software: Function get_company_as_string refactored to work after changes in data structure of companies founders. That was a simple approach of unit testing. In it I tried to explain the huge importance of that skill and help people that don’t know how to start in software testing. With unit tests, you can improve the quality of software. It’s very useful to prevent unwanted side-effects on maintenance of existing code. Unit Testing can prevent a lot of issue, like software crashing and waste of time with debugging. And helps you and your teammates to maintain an software with quality. All this, with easiness and not too much of work. I intended to talk a bit about TDD and Mock in this article, but, this article is already too big. So, it will be an another article in the future. If you found this article helpful, give me some claps 👏.

Unit Testing: A Simplistic and Language-Agnostic Approach

Being perfectly unbiased, and only using tools for any given job. If humans weren’t the entirely irrational collection of cells that we are, being technology agnostic wouldn’t even be a discussion — it would be taken for granted. the best Unfortunately, we people are just big bags flesh, bone and imperfection. Familiarity often overwhelms better judgement, which, if you’re a business leader, can have serious implications for the organisation that you steer. We spoke to three business leaders — , CIO of carsales.com, , CTO of , and , Head of Engineering at Woolworths Digital, about the need to be technology agnostic as a business leader, and their personal experiences working towards that aim. Jason Blackman Brett Raven RedBalloon David Bolton The Need for Agnosticism Blackman, Raven and Bolton each see being technically agnostic, at least at the leadership level, as a non-negotiable. They identify themselves as agnostic and have worked toward that end goal over a number of years. “As a senior leader I try to remain as technology agnostic as possible”, says Blackman. “The stage in my career that the transition [from being specialised] to being technically agnostic occurred when I personally made the decision to focus more on people leadership and head into more senior roles.” Bolton reiterates the statement — “I am technically agnostic, and I think that’s a prerequisite for responsible technology management. While having a deep understanding of a particular technology is always a plus, as a technology leader you have to realise that deep expertise can also generate weak spots. It can serve to blind you to upcoming changes in technology directions, plus all the peripheral elements around technology.” “I am definitely agnostic”, says Raven. He points out that moving into leadership removed some of the attached to working with particular tech — “Once I started working more directly at the architecture level, tech stacks became a set of tools rather than a camp that I belonged in.” tribal element The Wealth of Benefits So what exactly are the benefits of technology agnosticism? Blackman sums it up relatively succinctly — “The benefits are simply that the people I lead have the specialist knowledge — and are smarter than me in that regard — so being as agnostic as possible allows for new and innovative solutions to be presented that would have otherwise been disregarded by me through nothing more than unfamiliarity.” Raven sees agnosticism through the same lens. “It opens up more opportunities, and helps to keep my mind open to the ‘right tool for the job’, rather than just using the stack that I was most comfortable with.” Bolton agrees, pointing out that agnosticism allows you to be less invested in the , and more invested in the . “Being agnostic”, he surmises, “means that you’re better able to make consequential decisions.” approach outcome The Inherent Challenges But becoming technology agnostic isn’t exactly . A concerted effort is required to transition from the usual starting point of specialism; to toss aside familiarity in the search for the best way to do things. A level of trust is required of your team in order for this agnostic style of leadership to work. If you feel as though any team member has a vested interest in the selection of a particular technology it can serve to colour your decision on using it and can have you reverting back to that which you know. beer and skittles “The only downside to being agnostic is if the team you lead try and pull the wool over your eyes”, notes Blackman. “Having the right team –one which you are fully confident in and trust — is essential.” And once you become appropriately agnostic you need to work to retain it; to not fall back into old habits. The retention of agnosticism is far easier when you’re not working away at the programming coalface, but old habits traditionally die hard. The issue is that as the leader, you still need to have a deep understanding of the technologies being used, and familiarity with one particular course of action can bring out a latent bias even years down the track. “One does need deep expertise in something to know what questions to ask. So it’s more a point that being agnostic in the present means you’ve evolved to a point where you can apply your hard won principles across technologies and even disciplines, yet still be able to adapt your principles as the landscape changes”, points out Bolton. Retaining technology agnosticism requires more than a dash of self-discipline. Keeping on Top of New Tech A key part of being truly tech agnostic is to be constantly on the eye out for tools that work better than those you currently use. It can prove the difference between being two steps ahead of your competition, and two step behind. So how do our leaders go about staying up to date? “I keep an eye on channels dedicated to developers — StackOverflow and Quora primarily” declares Raven. “I also look to people like Martin Fowler, who spend their days thinking about these things.” Blackman also does a lot of reading, although is more than happy to admit that his team play the most important role in keeping him up to date. “They’re crucial in bringing me up to speed on new tech. It’s their passion for it, and my desire to connect, that keeps me actively seeking more knowledge.” Bolton prefers a similarly face-to-face approach. “I talk to people”, he says. “I find that having conversations to hear about the upsides and drawbacks of new technology, along with learning nuances and the ‘joy factor’, are much more important than reading about those technologies online, where everything looks great in the eyes of its proponents.” While it’s true that humans are inherently fallible, Blackman, Raven and Bolton prove that a firm focus on technology agnosticism can override our carnal urge to ignore the untried and tend toward that which we know. Being tech agnostic is certainly not easy, and constant work is required in order for this impartiality to be retained, but as these three gentlemen’s success shows, the pros of doing so outweigh the cons. overwhelmingly Looking to join a tech agnostic team? Check out our latest . job listings

Do Tech Leaders Need to Be Agnostic with Stacks?

Phil Andrews is a Cosmic JS Contributor. If you’re a developer and need to open up a new blog for yourself or a client, the idea of using WordPress probably makes you cringe. 64.5% of developers on StackOverflow stated that WordPress was one of their most dreaded platforms to work with. surveyed The dread makes sense. You can’t hand code an entire blog if you use WordPress for your content management system (CMS). You have to work within the WordPress stack and then use plugins for the extra pieces you need. This is antithetical to most people who write plain code. What you probably want is a system that lets you input your content, store your content, and fetch it in a manner that is straightforward and easy. If you can input your content in a WYSIWYG dashboard even better¹. While looking for something that did this I stumbled on . It turns out that Cosmic is exactly that, a programming language agnostic CMS. You can write your blog in whatever language you want for whatever platform you want. CosmicJS You put your content in and it gives you simple API endpoints to retrieve it from. One get request and your work is returned to you in a JSON object. Here’s how you can setup a CMS for your blog in no time flat… Once you’ve created and logged into your account go ahead and create a new bucket. Buckets are exactly what they sound like. A container that holds objects. The objects could all be of the same type or they could be different. Once you’ve created your bucket and you click on it you’ll notice a new menu on the sidebar. One of the options will be . Add Object Type This is how we add a new type of object to our bucket. We’re going to be creating a type object that will hold all of the content and metadata associated with each of our blog posts. Blog Post Once you’ve clicked and input a name into the name field note that the name has been mapped to an API endpoint. These endpoints, which you’ll find on every single object page in Cosmic, are the beauty to the system. This endpoint will return all of your type objects. Add Object Type Blog Post Make a simple get request to it and you’ll receive a JSON object that contains each of your posts and the metadata associated with each one. If you have 5 posts, it will return 5. 100, it will return 100. At it’s base every object in a bucket automatically includes a title field and an HTML input field. So right off the bat without adding any metafields you have the basic components of a blog post. To show more of the power Cosmic has lets add a cover image and description field to our . While you’re creating your new object, clicking on the Metafields Template tab at the top will bring up the UI to add these. Blog Object Now that we have our object type created we can create an actual object of that type. Once you’ve clicked you can Inside you’ll see the structure of the type object that we specified. All we need to do now is fill in each field with the content that we want. Blog Posts +Add Blog Post. Blog Post Once you’ve input your content and hit publish or save draft² you’ll be given an API endpoint. To retrieve just this post you would send a get request to that endpoint and you’ll get your entire pre-formatted post back in JSON form. Notice our added metafields and, most importantly, that our content remains in it’s HTML format. That’s all there is to it. All you have to do is build out the front-end of your blog in whatever specs you like, write some content, retrieve your content, and then inject that content wherever you need to. ¹Though you could opt to build out or implement your own editor. Cosmic’s API does include tools for that. ²The API endpoint will not be live if you save as a draft. Developer, entrepreneur, and writer. You can find me over at where I write about current projects, tech, code, and sometimes life itself. philandrews.io Article originally appears on https://philandrews.io/blog

Language Agnostic Content Management With Cosmic JS

Authors:
(1) Frank Palma Gomez, from Boston University and the work done by him and Ramon during their internship in Google Research and Google DeepMind respectively;
(2) Ramon Sanabria, The University of Edinburgh and the work done by him and Frank during their internship in Google Research and Google DeepMind respectively;
(3) Yun-hsuan Sung, Google Research;
(4) Daniel Cer, Google Research;
(5) Siddharth Dalmia, Google DeepMind and Equal Advising Contributions;
(6) Gustavo Hernandez Abrego, Google Research and Equal Advising Contributions. Authors: Authors: (1) Frank Palma Gomez, from Boston University and the work done by him and Ramon during their internship in Google Research and Google DeepMind respectively; (2) Ramon Sanabria, The University of Edinburgh and the work done by him and Frank during their internship in Google Research and Google DeepMind respectively; (3) Yun-hsuan Sung, Google Research; (4) Daniel Cer, Google Research; (5) Siddharth Dalmia, Google DeepMind and Equal Advising Contributions; (6) Gustavo Hernandez Abrego, Google Research and Equal Advising Contributions. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Method 2 Method 3 Data and Tasks 3 Data and Tasks 4 Model 4 Model 5 Experiments 5 Experiments 6 Related Work 6 Related Work 7 Conclusion 7 Conclusion 8 Acknowledgements and References 8 Acknowledgements and References A Appendix A Appendix 7 Conclusion We present an effective approach to developing a speech-to-text DE from a text-only LLM. Our findings suggest that by using a text-only LLM as a backbone model, we can drastically outperform previous approaches using considerably less speech-to-text training data. Additionally, we find that we can improve zero-shot speech translation by simply combining readily available translation and S2T data. We showcase our findings in 102 languages for S2T and 4 languages in S2TT; opening up the possibility of using speech-to-text DE’s in different cross-model and cross-lingual settings. 8 Acknowledgements We would like to thank Shankar Kumar and Ankur Bapna for the valuable feedback on the draft of the paper. Chris Tar, Mario Guajardo-Céspedes, and Jason Riesa for the early experiment discussions and feedback. Christian Frank, Duc Dung Nguyen, Alex Tudor, and Dalia El Badawy for helping answer questions about AudioPaLM. References Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M Tyers, and Gregor Weber. 2019. Common voice: A massivelymultilingual speech corpus. arXiv preprint arXiv:1912.06670. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33:12449–12460. Ankur Bapna, Colin Cherry, Yu Zhang, Ye Jia, Melvin Johnson, Yong Cheng, Simran Khanuja, Jason Riesa, and Alexis Conneau. 2022. mslam: Massively multilingual joint pre-training for speech and text. arXiv preprint arXiv:2202.01374. Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, et al. 2023. Audiolm: a language modeling approach to audio generation. IEEE/ACM Transactions on Audio, Speech, and Language Processing. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1–113. Yu-An Chung, Yu Zhang, Wei Han, Chung-Cheng Chiu, James Qin, Ruoming Pang, and Yonghui Wu. 2021. W2v-bert: Combining contrastive learning and masked language modeling for self-supervised speech pre-training. In 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 244–250. IEEE. Alexis Conneau, Min Ma, Simran Khanuja, Yu Zhang, Vera Axelrod, Siddharth Dalmia, Jason Riesa, Clara Rivera, and Ankur Bapna. 2023. Fleurs: Few-shot learning evaluation of universal representations of speech. In 2022 IEEE Spoken Language Technology Workshop (SLT), pages 798–805. IEEE. Paul-Ambroise Duquenne, Hongyu Gong, Ning Dong, Jingfei Du, Ann Lee, Vedanuj Goswani, Changhan Wang, Juan Pino, Benoît Sagot, and Holger Schwenk. 2022. Speechmatrix: A large-scale mined corpus of multilingual speech-to-speech translations. arXiv preprint arXiv:2211.04508. Paul-Ambroise Duquenne, Hongyu Gong, and Holger Schwenk. 2021. Multimodal and multilingual embeddings for large-scale speech mining. Advances in Neural Information Processing Systems, 34:15748– 15761. Paul-Ambroise Duquenne, Holger Schwenk, and Benoît Sagot. 2023. Sentence-level multimodal and language-agnostic representations. arXiv preprint arXiv:2308.11466. Google, Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023. Palm 2 technical report. arXiv preprint arXiv:2305.10403. Naman Goyal, Cynthia Gao, Vishrav Chaudhary, PengJen Chen, Guillaume Wenzek, Da Ju, Sanjan Krishnan, Marc’Aurelio Ranzato, Francisco Guzmán, and Angela Fan. 2021. The flores-101 evaluation benchmark for low-resource and multilingual machine translation. Transactions of the Association for Computational Linguistics, 10:522–538. Raia Hadsell, Sumit Chopra, and Yann LeCun. 2006. Dimensionality reduction by learning an invariant mapping. 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), 2:1735–1742. Michael Hassid, Tal Remez, Tu Anh Nguyen, Itai Gat, Alexis Conneau, Felix Kreuk, Jade Copet, Alexandre Defossez, Gabriel Synnaeve, Emmanuel Dupoux, et al. 2023. Textually pretrained speech language models. arXiv preprint arXiv:2305.13009. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:3451–3460. Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, et al. 2017. Google’s multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational Linguistics, 5:339–351. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. CoRR, abs/1412.6980. Kushal Lakhotia, Eugene Kharitonov, Wei-Ning Hsu, Yossi Adi, Adam Polyak, Benjamin Bolte, Tu-Anh Nguyen, Jade Copet, Alexei Baevski, Abdelrahman Mohamed, and Emmanuel Dupoux. 2021. On generative spoken language modeling from raw audio. Transactions of the Association for Computational Linguistics, 9:1336–1354. Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang. 2022. Sentence-t5: Scalable sentence encoders from pretrained text-to-text models. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1864–1874, Dublin, Ireland. Association for Computational Linguistics. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Brussels, Belgium. Association for Computational Linguistics. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International Conference on Machine Learning, pages 28492–28518. PMLR. Paul K Rubenstein, Chulayuth Asawaroengchai, Duc Dung Nguyen, Ankur Bapna, Zalán Borsos, Félix de Chaumont Quitry, Peter Chen, Dalia El Badawy, Wei Han, Eugene Kharitonov, et al. 2023. Audiopalm: A large language model that can speak and listen. arXiv preprint arXiv:2306.12925. Holger Schwenk, Vishrav Chaudhary, Shuo Sun, Hongyu Gong, and Francisco Guzmán. 2019. Wikimatrix: Mining 135m parallel sentences in 1620 language pairs from wikipedia. ArXiv, abs/1907.05791. Aditya Siddhant, Ankur Bapna, Orhan Firat, Yuan Cao, Mia Xu Chen, Isaac Caswell, and Xavier Garcia. 2022. Towards the next 1000 languages in multilingual machine translation: Exploring the synergy between supervised and self-supervised learning. arXiv preprint arXiv:2201.03110. Changhan Wang, Anne Wu, Jiatao Gu, and Juan Miguel Pino. 2021. Covost 2 and massively multilingual speech translation. In Interspeech. Changhan Wang, Anne Wu, and Juan Pino. 2020. Covost 2 and massively multilingual speech-to-text translation. arXiv preprint arXiv:2007.10310. Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. 2023. Neural codec language models are zero-shot text to speech synthesizers. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2020. mt5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934. Yinfei Yang, Gustavo Hernandez Abrego, Steve Yuan, Mandy Guo, Qinlan Shen, Daniel Cer, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2019. Improving multilingual sentence embedding using bidirectional dual encoder with additive margin softmax. arXiv preprint arXiv:1902.08564. Xu Zhang, Felix X. Yu, Sanjiv Kumar, and Shih-Fu Chang. 2017. Learning spread-out local feature descriptors. 2017 IEEE International Conference on Computer Vision (ICCV), pages 4605–4613. Yu Zhang, Wei Han, James Qin, Yongqiang Wang, Ankur Bapna, Zhehuai Chen, Nanxin Chen, Bo Li, Vera Axelrod, Gary Wang, et al. 2023. Google usm: Scaling automatic speech recognition beyond 100 languages. arXiv preprint arXiv:2303.01037. A Appendix A.1 Training Setup Ni et al. (2022) showed that applying a contrastive loss to sentence encoders leads to improved retrieval performance in downstream tasks. After initializing our model from the PaLM 2, we use a contrastive loss (Hadsell et al., 2006). Using equation 1, our multi-modal DE will learn from paired speech and text embeddings (xi , yi), where yi is considered as a positive example to xi while all other examples where i ̸= j are negative ones. The model should learn to bring the positive transcriptions closer to the corresponding speech sample, while pushing away all the other negative transcriptions. In our training, the positive and negative distinction is done within the training batch. Hence, we apply an in-batch softmax as part of our loss computation. Lastly, sim() is a similarity function formulated as the dot product between the speech sample and the transcription embeddings. To train our model, we use the sum of a contrastive loss with a spreadout loss (Zhang et al., 2017) of both the speech and text embeddings. We calculate the contrastive loss (Yang et al., 2019) in a bidirectional way, by adding the loss in the speech-to-text and the text-to-speech direction. A.2 Expressing Tasks For training and inference, we found that using a prefix improves speech-to-text retrieval performance. Therefore, we pre-pend a prefix containing the language and modality shown in in Table 3. In the case of a speech utterance, the prefix will be tokenized with the LLMs tokenizer and the remaining will be converted to audio tokens. A.3 Data Table 4 shows the training and evaluation datasets we used through out our experiments. We used 21 languages CoVoST-2 to train our model on speech-to-text retrieval which amounts to approximately 900 hours of speech. To evaluate our models speech-to-text retrieval capabilities, we evaluate on FLEURS speech-to-text test split on 102 languages. We use FLEURS speech-to-text translation test split to evaluate our models abilities on tasks that require cross-lingual and cross-modal knowledge. We evaluate of 4 different languages: German, Polish, French, and Dutch. We find that combining speech-to-text retrieval data and readily available translation data improves our models cross-lingual and cross-modal abilities. Table 5 shows the number of parallel sentences we used during training from X → En. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Stories, News & Mentions about AGNOSTIC

Agnostic’s Stories on HackerNoon